ArthurZ's picture
ArthurZ HF Staff
radar 2026-11-02
0e018ce verified
Raw History Blame Contribute Delete
284 kB
<!doctype html><meta charset=utf-8><title>transformers radar</title><style>body{max-width:1200px;margin:auto;padding:1em;font-family:system-ui}table{border-collapse:collapse;width:100%}td,th{border-bottom:1px solid #ddd;padding:4px 8px;text-align:left;vertical-align:top}code{font-size:85%}</style><h1>transformers downstream radar · 2026-11-02</h1>
<p>44 repos · last 2 days of GitHub activity · 662 monkey patches tracked</p>
<h2>Act on these</h2>
<table><tr><th></th><th>action</th><th>component</th><th>what</th><th>threads</th></tr><tr><td>🔴</td><td>fix in transformers</td><td>model:Qwen</td><td>Qwen3-VL vision features diverge from Transformers in fine-grained grounding</td><td><a href="https://github.com/sgl-project/sglang/issues/35772">sgl-project/sglang#35772</a></td></tr><tr><td>🔴</td><td>fix in transformers</td><td>tokenizers</td><td>GGUF conversion crashes when extra_special_tokens is not a dict</td><td><a href="https://github.com/ggml-org/llama.cpp/issues/31343">ggml-org/llama.cpp#31343</a></td></tr><tr><td>🔴</td><td>fix in transformers</td><td>model:Gemma</td><td>Gemma 4 + FP8 quantization causes image request crashes due to FP8-cast pixel values</td><td><a href="https://github.com/vllm-project/vllm/issues/63732">vllm-project/vllm#63732</a></td></tr><tr><td>🔴</td><td>fix in transformers</td><td>model:GLM</td><td>GLM-5.x under pipeline parallelism reuses layers with stale top-k indices</td><td><a href="https://github.com/vllm-project/vllm/issues/63170">vllm-project/vllm#63170</a></td></tr><tr><td>🔴</td><td>fix in transformers</td><td>model:DeepSeek</td><td>DeepSeek-V4 checkpoints with FP8 config incorrectly route through MXFP4 expert path</td><td><a href="https://github.com/sgl-project/sglang/issues/44813">sgl-project/sglang#44813</a></td></tr><tr><td>🔴</td><td>reply/guide users</td><td>from_pretrained</td><td>from_pretrained fails to load FAST action tokenizers from transformers 4</td><td><a href="https://github.com/huggingface/lerobot/issues/5155">huggingface/lerobot#5155</a></td></tr><tr><td>🔴</td><td>reply/guide users</td><td>model:Qwen</td><td>Qwen3.8-Flash-Next missing from_pretrained generation config</td><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4451">NVIDIA-NeMo/Automodel#4451</a></td></tr><tr><td>🔴</td><td>reply/guide users</td><td>model:GLM</td><td>GLM-5.3-Flash missing pretrained generation config in export</td><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4576">NVIDIA-NeMo/Automodel#4576</a></td></tr><tr><td>🟠</td><td>fix in transformers</td><td>quantization</td><td>BitsAndBytes 4-bit unmerge loses quantization scale, drifting base weights</td><td><a href="https://github.com/huggingface/peft/issues/4069">huggingface/peft#4069</a></td></tr><tr><td>🟠</td><td>fix in transformers</td><td>model:Qwen</td><td>Qwen3-MoE and Gemma 4 forced float32 patches not undone when flag turns off</td><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2325">unslothai/unsloth-zoo#2325</a></td></tr><tr><td>🟠</td><td>fix in transformers</td><td>tokenizers</td><td>tokenizer_config.json with TokenizersBackend class breaks loading in transformers&lt;5</td><td><a href="https://github.com/hiyouga/LlamaFactory/issues/10875">hiyouga/LlamaFactory#10875</a></td></tr><tr><td>🟠</td><td>reply/guide users</td><td>generate</td><td>SGLang detokenization degrades to O(n²) with SGLANG_SIMULATE_ACC_LEN</td><td><a href="https://github.com/sgl-project/sglang/issues/34740">sgl-project/sglang#34740</a></td></tr><tr><td>🟠</td><td>reply/guide users</td><td>model:Gemma</td><td>Gemma4Plugin audio processing fails due to missing None check in feature_extractor</td><td><a href="https://github.com/hiyouga/LlamaFactory/issues/10388">hiyouga/LlamaFactory#10388</a></td></tr><tr><td>⚪</td><td>reply/guide users</td><td>from_pretrained</td><td>PeftModelForQuestionAnswering silently drops token_type_ids and position_ids</td><td><a href="https://github.com/huggingface/peft/issues/3857">huggingface/peft#3857</a></td></tr><tr><td>⚪</td><td>reply/guide users</td><td>tokenizers</td><td>Error message for unmatched target_modules doesn&#x27;t reveal re.fullmatch usage</td><td><a href="https://github.com/huggingface/peft/issues/4067">huggingface/peft#4067</a></td></tr></table>
<details><summary>All 466 matching threads</summary>
<table><tr><th>thread</th><th>comments</th><th>title</th></tr><tr><td><a href="https://github.com/embeddings-benchmark/mteb/issues/4842">embeddings-benchmark/mteb#4842</a></td><td>217</td><td>[Research Project] MOEB: Massive Omni Embedding Benchmark</td></tr><tr><td><a href="https://github.com/huggingface/peft/issues/2310">huggingface/peft#2310</a></td><td>174</td><td>Comparison of Different Fine-Tuning Techniques for Conversational AI</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/50576">vllm-project/vllm#50576</a></td><td>125</td><td>[Feature]: SM8x (Ampere A100/A800) support for DeepSeek-V4-Flash / DeepSeek-V4-Flash-0731 DSpark</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/14730">unslothai/unsloth#14730</a></td><td>40</td><td>Studio: Flows, a node editor for AI pipelines, story games and 2D worlds</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/53298">vllm-project/vllm#53298</a></td><td>30</td><td>[Bugfix][KV Connector] Restore state after KV load rejection</td></tr><tr><td><a href="https://github.com/huggingface/lerobot/issues/5008">huggingface/lerobot#5008</a></td><td>30</td><td>feat(policies): add native Long-WAM training and inference</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/46091">sgl-project/sglang#46091</a></td><td>27</td><td>[Feat] usage: align prompt and cache accounting across APIs</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/41916">sgl-project/sglang#41916</a></td><td>25</td><td>[Fix] sampling: Make top-k/top-p filtering consistent across sampling paths</td></tr><tr><td><a href="https://github.com/huggingface/optimum/issues/351">huggingface/optimum#351</a></td><td>23</td><td>Add all available ONNX models to ORTConfigManager</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/63700">vllm-project/vllm#63700</a></td><td>21</td><td>[Bugfix][MoE] Avoid double router weighting in TRTLLM BF16 experts</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/56155">vllm-project/vllm#56155</a></td><td>20</td><td>[Core][Attention] Resolve per-layer KV dtype in attention runtime</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47229">sgl-project/sglang#47229</a></td><td>19</td><td>[Fix] kernels: Support OOT fused ops under torch.compile</td></tr><tr><td><a href="https://github.com/vllm-project/llm-compressor/issues/2808">vllm-project/llm-compressor#2808</a></td><td>19</td><td>AWQ GQA support</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16071">unslothai/unsloth#16071</a></td><td>18</td><td>LoRA: add the rank-r term inside the base GEMM so small updates are not rounded away</td></tr><tr><td><a href="https://github.com/verl-project/verl/issues/7483">verl-project/verl#7483</a></td><td>17</td><td>[vllm] feat: Support LoRA for DeepSeek V4 &amp; DeepSeek V4.1 &amp; GLM5.3 Flash</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/55947">vllm-project/vllm#55947</a></td><td>16</td><td>[Feature][Spec Decode] Add Uno shared-model drafting with a draft-only LoRA adapter (Model Runner V2)</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/57311">vllm-project/vllm#57311</a></td><td>13</td><td>[Perf][Startup] Move atomic_writer out of weight_utils so the model-info cache save stays light</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/31856">vllm-project/vllm#31856</a></td><td>13</td><td>[Bug]: Text degeneration / repetition loops with MiniMax-M2.1-NVFP4 on v0.14.0rc1.dev308</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/46858">sgl-project/sglang#46858</a></td><td>13</td><td>[Feat] lfm3: Add text, vision, audio and Omni support</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/36059">sgl-project/sglang#36059</a></td><td>13</td><td>[Perf] dsv4: Add native H16 CuTe sparse prefill backend</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/30771">sgl-project/sglang#30771</a></td><td>13</td><td>[Fix] tokenizer: Isolate and offload regular fast tokenization</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16032">unslothai/unsloth#16032</a></td><td>12</td><td>Opt-in fused batch-1 NF4 decode for Llama (UNSLOTH_DECODE_FUSED_NF4=1): 1.6-2.3x per token in eager generate</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16227">unslothai/unsloth#16227</a></td><td>12</td><td>Studio MLX: decode reply snapshots incrementally instead of re-decoding every id</td></tr><tr><td><a href="https://github.com/NVIDIA/TensorRT-LLM/issues/21251">NVIDIA/TensorRT-LLM#21251</a></td><td>12</td><td>[https://nvbugs/7011503][fix] Pass max_seq_len to MiniCPM-V 4.6 vision attention metadata</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/59693">vllm-project/vllm#59693</a></td><td>11</td><td>[ROCm][Perf] Kimi-K3: token-sharded residual stream for long prefills</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/7115">unslothai/unsloth#7115</a></td><td>11</td><td>Add EXL3 (ExLlamaV3) quantization backend: 2/3/4/6/8-bit and MoE support</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/45621">sgl-project/sglang#45621</a></td><td>11</td><td>[Feat] decisions: Serve StartLux-Decision checkpoints on /v1/systemone</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47503">sgl-project/sglang#47503</a></td><td>11</td><td>[Perf] rmsnorm: One SM100-tuned JIT template for plain, fused residual-add and fp8 per-tensor quant</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/33226">sgl-project/sglang#33226</a></td><td>11</td><td>[Feat] cp: Support FlashInfer prefill context parallelism</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/44883">sgl-project/sglang#44883</a></td><td>11</td><td>[Feat] [NPU] multimodal: Enable device-side multimodal feature transport via torch_npu IPC</td></tr><tr><td><a href="https://github.com/verl-project/verl/issues/7971">verl-project/verl#7971</a></td><td>11</td><td>[megatron, vllm, env] feat: add native NVFP4 training and rollout with live refit</td></tr><tr><td><a href="https://github.com/huggingface/peft/issues/3832">huggingface/peft#3832</a></td><td>11</td><td>FIX strict_adapter_check for load_adapter and from_pretrained</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4314">NVIDIA-NeMo/Automodel#4314</a></td><td>11</td><td>feat(glm5_next): packed CP/PP training and GLM-5.3-Flash kernel backends</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/61607">vllm-project/vllm#61607</a></td><td>10</td><td>[XPU] Add Qwen4Exp (Qwen3.8-Flash-Next) model architecture support</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/15993">unslothai/unsloth#15993</a></td><td>10</td><td>Studio: decode with CUDA graphs by default in the transformers chat backend</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16344">unslothai/unsloth#16344</a></td><td>10</td><td>Studio: wait on worker exit instead of fixed sleeps when switching models</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/14039">unslothai/unsloth#14039</a></td><td>10</td><td>Unsloth Studio (AMD): leave an iGPU out of the Train page VRAM total and the default GPU pick (blocks dual R9700 testing)</td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/issues/8602">deepspeedai/DeepSpeed#8602</a></td><td>10</td><td>feat(rollout): segment-based kernel injection for hybrid-engine greedy decode</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/37354">sgl-project/sglang#37354</a></td><td>10</td><td>[Feat] [Apple] mlx: Add an architecture-independent ATen-to-MLX lowering layer</td></tr><tr><td><a href="https://github.com/intel/auto-round/issues/2540">intel/auto-round#2540</a></td><td>10</td><td>refine export and format code and provide unify api</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/37191">vllm-project/vllm#37191</a></td><td>9</td><td>[Bug]: Benchmark tuning MoE kernel config fails due to triton 3.6</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64340">vllm-project/vllm#64340</a></td><td>9</td><td>[Bugfix][MoE] Preserve invalid expert IDs in DeepEP LL round-robin mapping</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/53739">vllm-project/vllm#53739</a></td><td>9</td><td>[Bugfix][Parser] Terminate streamed tool arguments the flush cannot extend</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16041">unslothai/unsloth#16041</a></td><td>9</td><td>Gemma 3: generate on the non-flex attention path instead of flex_attention&#x27;s fallback</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16073">unslothai/unsloth#16073</a></td><td>9</td><td>Flex attention on torch &lt; 2.14: fitting tiles for short decode calls so a batch-size change does not leave the process on eager attention</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16197">unslothai/unsloth#16197</a></td><td>9</td><td>Reset the inference stance counter in for_inference</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/46953">sgl-project/sglang#46953</a></td><td>9</td><td>[Feat] weight_cache: Add the vmm_fd transport</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/45792">sgl-project/sglang#45792</a></td><td>9</td><td>[Feat] [NPU] multimodal: Add JPEGD + VPC image decoding</td></tr><tr><td><a href="https://github.com/huggingface/peft/issues/3507">huggingface/peft#3507</a></td><td>9</td><td>PeftMixedModel.disable_adapter does not preserve nested or pre-disabled state</td></tr><tr><td><a href="https://github.com/NVIDIA/TensorRT-LLM/issues/20285">NVIDIA/TensorRT-LLM#20285</a></td><td>9</td><td>[https://nvbugs/6787519][fix] Initialize Nemotron-H-Puzzle configuration correctly</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/56116">vllm-project/vllm#56116</a></td><td>8</td><td>[Quantization] Support ModelOpt layer-wise mixed KV cache</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/38122">vllm-project/vllm#38122</a></td><td>8</td><td>[Bug]: Qwen 3.5 fails to load from GGUF</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/56327">vllm-project/vllm#56327</a></td><td>8</td><td>[LoRA][Model] Add LoRA support for GLM-5.3-Flash</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/39682">sgl-project/sglang#39682</a></td><td>8</td><td>[Feat] diffusion/model: support HiDream-O1-Image</td></tr><tr><td><a href="https://github.com/verl-project/verl/issues/7135">verl-project/verl#7135</a></td><td>8</td><td>[megatron] feat: support packed cu_seqlens</td></tr><tr><td><a href="https://github.com/huggingface/peft/issues/4069">huggingface/peft#4069</a></td><td>8</td><td>fix: reuse the 4-bit quantization scale when unmerging LoRA</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4208">NVIDIA-NeMo/Automodel#4208</a></td><td>8</td><td>refactor(speculative): share draft recipe lifecycle in a base class</td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/issues/8326">modelscope/ms-swift#8326</a></td><td>8</td><td>sft微调qwen3.5 构造自己的VQA数据集jsonl ,报错An error occurred while generating the dataset</td></tr><tr><td><a href="https://github.com/embeddings-benchmark/mteb/issues/5400">embeddings-benchmark/mteb#5400</a></td><td>8</td><td>Add public NGNN general text encoder with fixed 512D compression</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2296">unslothai/unsloth-zoo#2296</a></td><td>8</td><td>Reset the inference stance counter at every generate</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/63406">vllm-project/vllm#63406</a></td><td>7</td><td>[Core][CUDA Graph] Separate prefill (token) and decode (request) CUDA graph capture sizes</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64197">vllm-project/vllm#64197</a></td><td>7</td><td>[Feature] Support per-model Hugging Face and ModelScope sources</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/62667">vllm-project/vllm#62667</a></td><td>7</td><td>[Frontend] Support sanitize_control_tokens in chat templates</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/61542">vllm-project/vllm#61542</a></td><td>7</td><td>[Bugfix][Tool Parser] Preserve whitespace in HYV4 argument names</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/56218">vllm-project/vllm#56218</a></td><td>7</td><td>[Bugfix][Model] Don&#x27;t carry realtime context into single-turn prompt models</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/27983">sgl-project/sglang#27983</a></td><td>7</td><td>[Fix] hf_transformers: Apply rope overrides to text_config for VLM-format models</td></tr><tr><td><a href="https://github.com/huggingface/lerobot/issues/4626">huggingface/lerobot#4626</a></td><td>7</td><td>[Bug/Feature Req/Fix?] DAgger rollout strategy cannot record full task-attempt episodes with HIL corrections</td></tr><tr><td><a href="https://github.com/ml-explore/mlx-lm/issues/1261">ml-explore/mlx-lm#1261</a></td><td>7</td><td>Support for Zyphra/ZAYA1-base</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2243">unslothai/unsloth-zoo#2243</a></td><td>7</td><td>Vendored fla: faster KDA decode, masked KDA output-kernel load, cached backend probe (fla #1415 #1416 #1582 #996)</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/47548">vllm-project/vllm#47548</a></td><td>6</td><td>[Bug]: Combination of sleep mode and speculative decoding cause crash (ROCm / MI250)</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/52630">vllm-project/vllm#52630</a></td><td>6</td><td>[Performance]: ~18% pooling throughput regression starting with torch 2.11 (vLLM v0.20.0+)</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/63170">vllm-project/vllm#63170</a></td><td>6</td><td>[Bug]: GLM-5.x (DSA) under pipeline parallelism: reuse layers at a stage start read stale top-k indices</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64282">vllm-project/vllm#64282</a></td><td>6</td><td>[Model] Add Nemotron-3.5-Transcribe</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/37220">vllm-project/vllm#37220</a></td><td>6</td><td>[Bugfix] Consolidate Gemma2/3 GGUF fixes for correctness on Blackwell</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/43408">vllm-project/vllm#43408</a></td><td>6</td><td>[Feature] Tolerate malformed Harmony streams.</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16317">unslothai/unsloth#16317</a></td><td>6</td><td>[Bug] Unsloth North Mini Code giving Stream interrupted error when using Bearer Token and OpenAIAPI with OpenCode</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/15934">unslothai/unsloth#15934</a></td><td>6</td><td>cuDNN SDPA: keep known-bad decode and NaN-dQ shapes off cuDNN before torch 2.14 / 2.15</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16049">unslothai/unsloth#16049</a></td><td>6</td><td>Flex attention: small shared-memory tiles for short no-grad calls so one short prompt does not leave the process on eager attention</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47785">sgl-project/sglang#47785</a></td><td>6</td><td>[Perf] [NVIDIA] qwen3: Fuse prefill QK norm and RoPE with contiguous Q</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/33059">sgl-project/sglang#33059</a></td><td>6</td><td>[Fix] [AMD] dsa: Align Aiter paged rows with DP-attention metadata</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47271">sgl-project/sglang#47271</a></td><td>6</td><td>[Feat] Mistral-Large-4: add model support</td></tr><tr><td><a href="https://github.com/huggingface/lerobot/issues/4634">huggingface/lerobot#4634</a></td><td>6</td><td>perf(smolvla): static immutable prefix KV cache and torch.compile support (4.2x speedup)</td></tr><tr><td><a href="https://github.com/verl-project/verl/issues/8398">verl-project/verl#8398</a></td><td>6</td><td>[rollout, vllm] fix: keep vLLM&#x27;s MXFP4 MoE selection on Hopper instead of forcing deep_gemm</td></tr><tr><td><a href="https://github.com/NVIDIA/TensorRT-LLM/issues/20853">NVIDIA/TensorRT-LLM#20853</a></td><td>6</td><td>[None][feat] Classify startup failures, surface their root cause and report them in telemetry</td></tr><tr><td><a href="https://github.com/intel/auto-round/issues/2457">intel/auto-round#2457</a></td><td>6</td><td>fix: disable trust_remote_code by default for security</td></tr><tr><td><a href="https://github.com/intel/auto-round/issues/2546">intel/auto-round#2546</a></td><td>6</td><td>fix: register copied-through MTP modules in compressed-tensors ignore list</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2240">unslothai/unsloth-zoo#2240</a></td><td>6</td><td>GLM-5.3-Flash DSA: exact skip, chunked pooled indexer, opt-in flex BlockMask; free failed flex calls&#x27; tensors</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/61267">vllm-project/vllm#61267</a></td><td>5</td><td>[Bugfix][LoRA] respect `attention_k_eq_v=True` QKK LoRA layers</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/50616">vllm-project/vllm#50616</a></td><td>5</td><td>[Bug]: runai_streamer multi-node: aux (non-tensor) files are not pulled on peer nodes</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/65034">vllm-project/vllm#65034</a></td><td>5</td><td>[Bug]: Quark `exclude` entries written as weight names (e.g. `mtp.fc.weight`) are silently never matched, breaking MTP speculative decoding</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/45978">vllm-project/vllm#45978</a></td><td>5</td><td>[Bugfix] Honor override_generation_config for special tokens</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/38606">vllm-project/vllm#38606</a></td><td>5</td><td>[Bug]: KV block corruption under rapid LoRA adapter alternation</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/37159">vllm-project/vllm#37159</a></td><td>5</td><td>[Bug]: vLLM crashed with V100 by running zai-org/GLM-OCR</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/37261">vllm-project/vllm#37261</a></td><td>5</td><td>[Bug]: KeyError when use lmcache layerwise mode in vllm 0.14.0</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/61524">vllm-project/vllm#61524</a></td><td>5</td><td>fix(renderers): preserve special tokens when truncation_side matches tokenizer default</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/62162">vllm-project/vllm#62162</a></td><td>5</td><td>[Feature][KV Offload] Enable canonical offload for chunked-local attention</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/15976">unslothai/unsloth#15976</a></td><td>5</td><td>Decode CUDA graphs: opt-in whole-step capture for batch 1 (UNSLOTH_DECODE_CUDA_GRAPHS_FULL=1)</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16265">unslothai/unsloth#16265</a></td><td>5</td><td>Studio RAG: count chunking tokens in batches</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16266">unslothai/unsloth#16266</a></td><td>5</td><td>Studio export: upload a reused folder&#x27;s fresh files instead of saving the model twice</td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/issues/8777">deepspeedai/DeepSpeed#8777</a></td><td>5</td><td>[RFC] Backend-pluggable low-precision training for Dense Linear and AutoEP grouped GEMM (Ascend HiFloat8)</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/37216">sgl-project/sglang#37216</a></td><td>5</td><td>[Bug] remote_instance weight loading fails with --pp-size 2 — TransferEngine info keyed by tp_rank</td></tr><tr><td><a href="https://github.com/huggingface/diffusers/issues/15131">huggingface/diffusers#15131</a></td><td>5</td><td>Flux2KleinInpaintPipeline: `padding_mask_crop` pastes a center crop of the result instead of resizing it</td></tr><tr><td><a href="https://github.com/huggingface/lerobot/issues/5148">huggingface/lerobot#5148</a></td><td>5</td><td>[Performance] pi0.5: the denoise loop is 86% of a bf16 chunk; static prefix KV + CUDA-graph denoise?</td></tr><tr><td><a href="https://github.com/huggingface/lerobot/issues/5163">huggingface/lerobot#5163</a></td><td>5</td><td>fix(policies): let LingBot-VLA v2, the LaWAM VLM stage and the GR00T N1.7 backbone pass torch.export</td></tr><tr><td><a href="https://github.com/huggingface/peft/issues/3830">huggingface/peft#3830</a></td><td>5</td><td>OFT module_dropout has had no effect since #2575: the MultiplicativeDropoutLayer is created but never called</td></tr><tr><td><a href="https://github.com/huggingface/peft/issues/3942">huggingface/peft#3942</a></td><td>5</td><td>FIX Support MoE expert target_parameters with TP/EP</td></tr><tr><td><a href="https://github.com/huggingface/trl/issues/7618">huggingface/trl#7618</a></td><td>5</td><td>SDFT and SDPO build the teacher in fp32 with the default attention when the model is passed as an instance</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4534">NVIDIA-NeMo/Automodel#4534</a></td><td>5</td><td>fix(deepseek-v4): keep the text-only forward CUDA-graph capturable</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4514">NVIDIA-NeMo/Automodel#4514</a></td><td>5</td><td>perf(moe): add configurable FSDP block prefetch</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4542">NVIDIA-NeMo/Automodel#4542</a></td><td>5</td><td>feat(glm5_next): train and export the GLM-5.3-Flash MTP layer</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4454">NVIDIA-NeMo/Automodel#4454</a></td><td>5</td><td>fix(glm5_next): keep layer split and architectures in exported config</td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/issues/8314">modelscope/ms-swift#8314</a></td><td>5</td><td>Qwen 3.5 35B A3B SFT example perf issue</td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/issues/9593">modelscope/ms-swift#9593</a></td><td>5</td><td>如何重构dataset的默认getitem函数</td></tr><tr><td><a href="https://github.com/vllm-project/llm-compressor/issues/3418">vllm-project/llm-compressor#3418</a></td><td>5</td><td>Fix distributed oneshot calibrating every rank on the same rows</td></tr><tr><td><a href="https://github.com/embeddings-benchmark/mteb/issues/5652">embeddings-benchmark/mteb#5652</a></td><td>5</td><td>fix: video frame timing</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2325">unslothai/unsloth-zoo#2325</a></td><td>5</td><td>Qwen3-MoE / Gemma 4: undo the forced float32 patches when a later load turns the flag off</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/65005">vllm-project/vllm#65005</a></td><td>4</td><td>[Docs][Model] Add Parakeet TDT transcription support via plugin</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/63909">vllm-project/vllm#63909</a></td><td>4</td><td>[Bugfix][Spec Decode] Propagate dictionary hf-overrides to speculative draft models</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/50577">vllm-project/vllm#50577</a></td><td>4</td><td>[Bug]: AxionML/Gemma-4-12B-NVFP4 fails with shape mismatch in flashinfer_scaled_fp4_mm on RTX 5060 Ti (sm_120)</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64874">vllm-project/vllm#64874</a></td><td>4</td><td>[Bug]: [ROCm][GLM-5.3-Flash]: &quot;Workspace is locked&quot; AssertionError in rocm_fp8_paged_mqa_logits under high decode concurrency (glm5next AMD indexer reserves decode logits via torch.empty, not the workspace)</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64959">vllm-project/vllm#64959</a></td><td>4</td><td>[Model] Add LiquidAI LFM3 (Lfm3ForCausalLM)</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64326">vllm-project/vllm#64326</a></td><td>4</td><td>[Bug]: DelegatingParser.parse ignores model_output_token_ids, so markers written as plain text become tool calls or end reasoning (non-streaming)</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/63732">vllm-project/vllm#63732</a></td><td>4</td><td>[Bug]: Gemma 4 + online --quantization fp8: image requests crash since v0.23.0 (FP8-cast pixel values reach quantized patch_embedder.input_proj); v0.22.0 works</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/37185">vllm-project/vllm#37185</a></td><td>4</td><td>[Bug]: The 2-bit model repeatedly outputs !!!!, while Transformers works correctly.</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16365">unslothai/unsloth#16365</a></td><td>4</td><td>feat(studio): system-wide voice typing with on-device grammar</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16293">unslothai/unsloth#16293</a></td><td>4</td><td>SFT: answer the pre-tokenized cap check from Arrow lengths when they prove it</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16189">unslothai/unsloth#16189</a></td><td>4</td><td>Import: scan installed distributions once while transformers and TRL import</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16203">unslothai/unsloth#16203</a></td><td>4</td><td>RL: read package versions once per trainer patch pass</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16243">unslothai/unsloth#16243</a></td><td>4</td><td>Studio: chat with LFM3-Audio-2B (text and speech in, text out)</td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/issues/10388">hiyouga/LlamaFactory#10388</a></td><td>4</td><td>fix: add None check for feature_extractor in Gemma4Plugin audio processing</td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/issues/8501">deepspeedai/DeepSpeed#8501</a></td><td>4</td><td>Fix ZeRO-3 checkpoint tag selection during weight reconstruction</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47754">sgl-project/sglang#47754</a></td><td>4</td><td>[Fix] multimodal: Make CUDA VMM feature transport opt-in</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/41110">sgl-project/sglang#41110</a></td><td>4</td><td>[Feature] Qwen3.5forsequenceclassification model is supported</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/35772">sgl-project/sglang#35772</a></td><td>4</td><td>[Bug] Qwen3-VL vision features diverge from Transformers/vLLM in v0.5.17 on fine-grained grounding</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47103">sgl-project/sglang#47103</a></td><td>4</td><td>[Perf] scheduler: Add opt-in stop-string tail reuse and prefix pruning</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/44435">sgl-project/sglang#44435</a></td><td>4</td><td>[Feat] [NPU] multimodal: Add opt-in cross-request ViT image batching</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/34740">sgl-project/sglang#34740</a></td><td>4</td><td>[Bug] SGLANG_SIMULATE_ACC_LEN silently degrades detokenization to O(n²) — `predict.fill_(100)` emits a byte-fallback token, and the `endswith(&quot;\ufffd&quot;)` commit gate then never advances the incremental-detokenization offsets</td></tr><tr><td><a href="https://github.com/huggingface/lerobot/issues/5106">huggingface/lerobot#5106</a></td><td>4</td><td>fix(pi0): mask padded actions in the loss</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4434">NVIDIA-NeMo/Automodel#4434</a></td><td>4</td><td>feat(gemma4_unified): model-owned context parallelism</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4571">NVIDIA-NeMo/Automodel#4571</a></td><td>4</td><td>fix(qwen): port HF packed fallback boundary guards</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4583">NVIDIA-NeMo/Automodel#4583</a></td><td>4</td><td>fix(deepseek): add fixed-order TileLang sparse attention backward</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4481">NVIDIA-NeMo/Automodel#4481</a></td><td>4</td><td>fix(attention): expose backend selection and document precedence</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4451">NVIDIA-NeMo/Automodel#4451</a></td><td>4</td><td>fix(qwen3.8): restore pretrained generation config</td></tr><tr><td><a href="https://github.com/huggingface/accelerate/issues/4330">huggingface/accelerate#4330</a></td><td>4</td><td>Resuming with load_state + skip_first_batches (checkpointing example) does not restore the shuffle order</td></tr><tr><td><a href="https://github.com/vllm-project/llm-compressor/issues/3408">vllm-project/llm-compressor#3408</a></td><td>4</td><td>Share NVFP4 global scales across Qwen3.5 gated delta net projections</td></tr><tr><td><a href="https://github.com/embeddings-benchmark/mteb/issues/5822">embeddings-benchmark/mteb#5822</a></td><td>4</td><td>Add Etsu-Embedding-0.8B-Memory</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2324">unslothai/unsloth-zoo#2324</a></td><td>4</td><td>train_on_responses_only: find fully masked rows from the Arrow labels column</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2310">unslothai/unsloth-zoo#2310</a></td><td>4</td><td>Tests: offline merge checks reset hub 0.x sessions so earlier online tests cannot leak</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2195">unslothai/unsloth-zoo#2195</a></td><td>4</td><td>gpt-oss: preserve left-padding masks in training</td></tr><tr><td><a href="https://github.com/ggml-org/llama.cpp/issues/30670">ggml-org/llama.cpp#30670</a></td><td>3</td><td>model: add Beam (Reflection)</td></tr><tr><td><a href="https://github.com/ggml-org/llama.cpp/issues/31471">ggml-org/llama.cpp#31471</a></td><td>3</td><td>mtmd : add Nemotron 3.5 Transcribe support</td></tr><tr><td><a href="https://github.com/ggml-org/llama.cpp/issues/31343">ggml-org/llama.cpp#31343</a></td><td>3</td><td>gguf-py : fix crash when extra_special_tokens is not a dict</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/48076">vllm-project/vllm#48076</a></td><td>3</td><td>[Performance]: Triton attention drops long-context decode from 3D split-KV to 2D at batch ≥ ~12, ~doubling ITL</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/49031">vllm-project/vllm#49031</a></td><td>3</td><td>[Bug]: FlashInfer B12x W4A16 runtime repacking retains original and packed MoE weights, causing startup OOM</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/50557">vllm-project/vllm#50557</a></td><td>3</td><td>[Bug]: MiniMax-M3-NVFP4 illegal memory access</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/50433">vllm-project/vllm#50433</a></td><td>3</td><td>[Performance][MooncakeStoreConnector] Overlapping async cold-prefix loads cause duplicate Gets into distinct GPU blocks</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/46611">vllm-project/vllm#46611</a></td><td>3</td><td>[Bug]: No available shared memory broadcast block found in 60 seconds. This typically happens when some processes are hanging or doing some time-consuming work (e.g. compilation, weight/kv cache quantization).</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64119">vllm-project/vllm#64119</a></td><td>3</td><td>[Fast Start] Support ModelOpt CUTLASS block FP8 cache restarts</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/59206">vllm-project/vllm#59206</a></td><td>3</td><td>[Transformers backend] Migrate ModernBERT embedding, reranker and token classification models</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/61320">vllm-project/vllm#61320</a></td><td>3</td><td>Log EAGLE draft rope base at startup; warn on silent default</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/63956">vllm-project/vllm#63956</a></td><td>3</td><td>[Perf][DSA] Token-sharded sparse-MLA prefill under TP; remove indexer row sharding</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/58379">vllm-project/vllm#58379</a></td><td>3</td><td>[Bugfix][Multimodal] Reject empty Qwen2-VL video samples</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64882">vllm-project/vllm#64882</a></td><td>3</td><td>[Bug]: Multimodal video loaders ignore --media-io-kwargs num_frames</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64536">vllm-project/vllm#64536</a></td><td>3</td><td>[Feature][Structured Output] Support continued assistant prefixes</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/62160">vllm-project/vllm#62160</a></td><td>3</td><td>[Bugfix][KV Offload] Deduplicate replicated GQA heads in canonical storage</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16246">unslothai/unsloth#16246</a></td><td>3</td><td>Tests: gpt-oss dense-expert reload fixture drops replaced experts&#x27; quant state</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16178">unslothai/unsloth#16178</a></td><td>3</td><td>[Bug] Studio: image generation returns 404 after a managed account loads a cached gated model</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16310">unslothai/unsloth#16310</a></td><td>3</td><td>Studio diffusion: change LoRA weights without reloading the adapter files</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16186">unslothai/unsloth#16186</a></td><td>3</td><td>[Bug] [Critical Regression / Severity: Incident-level bug] Isolated install via UNSLOTH_STUDIO_HOME silently overwrites existing baseline venv</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16373">unslothai/unsloth#16373</a></td><td>3</td><td>Notebooks CI: refresh the Colab oracle snapshots to the current GPU runtime</td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/issues/10875">hiyouga/LlamaFactory#10875</a></td><td>3</td><td>[export] keep exported tokenizer_config loadable with transformers&lt;5</td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/issues/8980">deepspeedai/DeepSpeed#8980</a></td><td>3</td><td>[Hybrid Engine] Require an explicit accelerator opt-in for graph capture</td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/issues/8939">deepspeedai/DeepSpeed#8939</a></td><td>3</td><td>feat(rollout): non-greedy sampling on hybrid engine graph decode path</td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/issues/8933">deepspeedai/DeepSpeed#8933</a></td><td>3</td><td>[Hybrid Engine] Enable graph capture for HybridEngine on non-CUDA accelerators</td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/issues/8659">deepspeedai/DeepSpeed#8659</a></td><td>3</td><td>Add deepspeed.models.glm_moe_dsa: TileLang sparse attention for GLM-5.2</td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/issues/8739">deepspeedai/DeepSpeed#8739</a></td><td>3</td><td>Overlap AutoEP&#x27;s expert weight gradients with the DeepEP backward combine</td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/issues/8959">deepspeedai/DeepSpeed#8959</a></td><td>3</td><td>Honor explicit dtype in ZeRO tensor constructors</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47367">sgl-project/sglang#47367</a></td><td>3</td><td>[Feat] [NPU] attention: support mixed-chunk on the ascend attention backend</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47129">sgl-project/sglang#47129</a></td><td>3</td><td>[Perf] [NVIDIA] muse-spark: Add B200 runtime optimizations</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47769">sgl-project/sglang#47769</a></td><td>3</td><td>[Feat] multimodal: Add bounded GPU feature publication admission</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/37753">sgl-project/sglang#37753</a></td><td>3</td><td>[Do not merge - need to clean] GLM-5.3-Flash (glm5_next) LoRA serving + glm53next port rebased</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/41403">sgl-project/sglang#41403</a></td><td>3</td><td>[Benchmark] Add synthetic audio input to bench_serving</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/46483">sgl-project/sglang#46483</a></td><td>3</td><td>[Perf] [NPU] attention: Avoid CPU sequence length synchronization for Kimi K3</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/44813">sgl-project/sglang#44813</a></td><td>3</td><td>[Fix] deepseek_v4: route unquantized BF16/F16/F32 checkpoints off the MXFP4 expert path</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/37183">sgl-project/sglang#37183</a></td><td>3</td><td>AMD MI308X SGLang GLM-5.3-Flash ValueError: The checkpoint you are trying to load has model type `glm5_next` but Transformers does not recognize this architecture.</td></tr><tr><td><a href="https://github.com/huggingface/diffusers/issues/15124">huggingface/diffusers#15124</a></td><td>3</td><td>Quantizer configs are mutated in place, leaking modules_to_not_convert across loads</td></tr><tr><td><a href="https://github.com/huggingface/diffusers/issues/15144">huggingface/diffusers#15144</a></td><td>3</td><td>Fix/flux lora quantization compute embeddings</td></tr><tr><td><a href="https://github.com/huggingface/diffusers/issues/15148">huggingface/diffusers#15148</a></td><td>3</td><td>[core] Enforce minimum supported versions in availability checks</td></tr><tr><td><a href="https://github.com/huggingface/diffusers/issues/15147">huggingface/diffusers#15147</a></td><td>3</td><td>Fix replicated parameter dtypes during tensor-parallel loading</td></tr><tr><td><a href="https://github.com/huggingface/lerobot/issues/5155">huggingface/lerobot#5155</a></td><td>3</td><td>fix(policies): load FAST action tokenizers saved by transformers 4</td></tr><tr><td><a href="https://github.com/huggingface/lerobot/issues/4778">huggingface/lerobot#4778</a></td><td>3</td><td>feat(policies): load GR00T fine-tunes without NVIDIA&#x27;s base model</td></tr><tr><td><a href="https://github.com/huggingface/lerobot/issues/5152">huggingface/lerobot#5152</a></td><td>3</td><td>perf(pi05): share one read-only prefix cache across the denoise steps</td></tr><tr><td><a href="https://github.com/huggingface/lerobot/issues/5151">huggingface/lerobot#5151</a></td><td>3</td><td>feat(fastwam): add Streaming-WAM to Fast-WAM (2.9x faster warm inference on L40S, no pause at chunk refresh)</td></tr><tr><td><a href="https://github.com/huggingface/peft/issues/4050">huggingface/peft#4050</a></td><td>3</td><td>fix: report global dims of tensor-parallel LoRA adapters</td></tr><tr><td><a href="https://github.com/huggingface/peft/issues/4067">huggingface/peft#4067</a></td><td>3</td><td>Error message for unmatched target_modules does not reveal that patterns are matched with re.fullmatch</td></tr><tr><td><a href="https://github.com/huggingface/peft/issues/3857">huggingface/peft#3857</a></td><td>3</td><td>PeftModelForQuestionAnswering silently drops `token_type_ids` and `position_ids`</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4575">NVIDIA-NeMo/Automodel#4575</a></td><td>3</td><td>feat(glm): FP8 SFT recipe for GLM-5.3 with model-owned lm_head/indexer precision</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4585">NVIDIA-NeMo/Automodel#4585</a></td><td>3</td><td>fix(perf): recognize B300 and GB300 for MFU</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4543">NVIDIA-NeMo/Automodel#4543</a></td><td>3</td><td>feat(deepseek_v41): part-by-part FP8/FP4 base-model loading</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4576">NVIDIA-NeMo/Automodel#4576</a></td><td>3</td><td>fix(glm5_next): restore pretrained generation config</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4565">NVIDIA-NeMo/Automodel#4565</a></td><td>3</td><td>feat(deepseek-v41): add packed vision CP and 128K recipe</td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/issues/8336">modelscope/ms-swift#8336</a></td><td>3</td><td>使用Lora微调时,检查点保存时出现adapter name 不匹配</td></tr><tr><td><a href="https://github.com/NVIDIA/TensorRT-LLM/issues/20973">NVIDIA/TensorRT-LLM#20973</a></td><td>3</td><td>[None][perf] Move vision pixel normalization from CPU to GPU</td></tr><tr><td><a href="https://github.com/NVIDIA/TensorRT-LLM/issues/20907">NVIDIA/TensorRT-LLM#20907</a></td><td>3</td><td>[TRTLLM-16460][feat] agent-flow: unified trtllm_modeling_v2 workflow for gated ModelingV2 onboarding and perf tuning</td></tr><tr><td><a href="https://github.com/NVIDIA/TensorRT-LLM/issues/20714">NVIDIA/TensorRT-LLM#20714</a></td><td>3</td><td>[TRTLLM-17238][feat] Add PrimTS sparse core backend for Qwen3.8 QSA attention</td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/issues/4230">axolotl-ai-cloud/axolotl#4230</a></td><td>3</td><td>feat(kernels): Mistral Large 4 fused MLA attention LoRA + block-FP8 training on transformers 5.22</td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/issues/4227">axolotl-ai-cloud/axolotl#4227</a></td><td>3</td><td>fix: follow kernels 0.18&#x27;s per-module kernelized function copies</td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/issues/4231">axolotl-ai-cloud/axolotl#4231</a></td><td>3</td><td>fix: kernelize device for kernels 0.18, tree-recurrence fused Mamba2 on odd row widths</td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/issues/4232">axolotl-ai-cloud/axolotl#4232</a></td><td>3</td><td>feat(kernels): model-agnostic grouped-expert kernels (block-FP8 / MXFP8 / Marlin dX) + moe_grouped_mode auto</td></tr><tr><td><a href="https://github.com/ml-explore/mlx-lm/issues/2018">ml-explore/mlx-lm#2018</a></td><td>3</td><td>Add Mistral Large 4 text model</td></tr><tr><td><a href="https://github.com/intel/auto-round/issues/2564">intel/auto-round#2564</a></td><td>3</td><td>fix: support nested Qwen2.5-Omni thinker layout</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2358">unslothai/unsloth-zoo#2358</a></td><td>3</td><td>GLM5 next sparse flex grads test: scale atol with each parameter gradient</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2357">unslothai/unsloth-zoo#2357</a></td><td>3</td><td>MoE expert LoRA vanilla load test: run on transformers 4.x, and mark the legacy Qwen3-VL-MoE packing it loads silently wrong</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2356">unslothai/unsloth-zoo#2356</a></td><td>3</td><td>MoE expert LoRA vanilla load test: skip the two checks PEFT &lt; 0.20 fails without Unsloth</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2354">unslothai/unsloth-zoo#2354</a></td><td>3</td><td>GLM5 next tests: keep assert_close mismatch report in the gradient checks</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2340">unslothai/unsloth-zoo#2340</a></td><td>3</td><td>rand4x cache tag: import set_torch_config absolutely</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2338">unslothai/unsloth-zoo#2338</a></td><td>3</td><td>GLM-5.3-Flash flex: keep eager calls with an empty real-query selection dense</td></tr><tr><td><a href="https://github.com/ggml-org/llama.cpp/issues/31456">ggml-org/llama.cpp#31456</a></td><td>2</td><td>gptneox: FFN uses tanh GELU (LLM_FFN_GELU) but GPT-NeoX/Pythia are trained with exact erf GELU</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/63369">vllm-project/vllm#63369</a></td><td>2</td><td>[Multimodal] Add TorchCodec image decoding backend</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/59058">vllm-project/vllm#59058</a></td><td>2</td><td>[Bugfix][MoRIIO] Preserve WRITE receive mappings after consumer abort</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64412">vllm-project/vllm#64412</a></td><td>2</td><td>[Bugfix][Frontend] Pool HF fast tokenizers in all renderers</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64543">vllm-project/vllm#64543</a></td><td>2</td><td>[Bugfix][Model] Fix dots_ocr vision tower QKV split for TP &gt; 1</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/62395">vllm-project/vllm#62395</a></td><td>2</td><td>[Feat][2/2] Add Transformers TPSP dense baseline support</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/47528">vllm-project/vllm#47528</a></td><td>2</td><td>[Bug]: DeepSeek-V4-Pro (DeepseekV4ForCausalLM, scale_fmt=ue8m0 / FP4 MoE) produces garbled / degenerate output under tensor parallelism (TP), while data parallelism + expert parallelism (DP+EP) works correctly</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/62752">vllm-project/vllm#62752</a></td><td>2</td><td>[Performance]: Avoidable preemption at hybrid Attention/Mamba decode block transitions</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/61888">vllm-project/vllm#61888</a></td><td>2</td><td>[Distributed] Opt-in FP16 partial sums for tensor-parallel row-parallel layers</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/63636">vllm-project/vllm#63636</a></td><td>2</td><td>[Bugfix][Transformers] Infer mm encoder dynamic_arg_dims from forward signature</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/65021">vllm-project/vllm#65021</a></td><td>2</td><td>[Doc][ROCm] Add verified MI300X and MI325X entries to the Qwen3.8-27B…</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/60608">vllm-project/vllm#60608</a></td><td>2</td><td>[Bug]: Exllama GPTQ and RDNA3 W4A16 kernels silently produce wrong output when `group_size % 32 != 0` (reproduced on CUDA sm_120 and ROCm gfx1100)</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/56180">vllm-project/vllm#56180</a></td><td>2</td><td>Failed: Cuda error /workspace/csrc/custom_all_reduce.cuh:164 &#x27;invalid argument&#x27;</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64239">vllm-project/vllm#64239</a></td><td>2</td><td>[Bug]: Timed-out overflow dequeue returns the wrong message</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64968">vllm-project/vllm#64968</a></td><td>2</td><td>[WIP][Feature] Qwen GDN BI + MTP + prefix cache integration</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/61353">vllm-project/vllm#61353</a></td><td>2</td><td>[Bugfix][Spec Decode] Fix DFlash capture on FLEX_ATTENTION and honor drafter enforce_eager</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/47326">vllm-project/vllm#47326</a></td><td>2</td><td>[Bug]: Incoherent output depending on tensor-parallel-size during inference on Leanstral (Mistral Small 4 family)</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64899">vllm-project/vllm#64899</a></td><td>2</td><td>[Bugfix][Multimodal] Honour explicit num_frames in Qwen-VL video loaders</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16340">unslothai/unsloth#16340</a></td><td>2</td><td>CLI: inference and chat cannot take an image, so a VLM can be trained but not used</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16347">unslothai/unsloth#16347</a></td><td>2</td><td>CLI: inference and chat take images for vision models</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16212">unslothai/unsloth#16212</a></td><td>2</td><td>Studio: check the weight index of a module path that only reads as the root</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16242">unslothai/unsloth#16242</a></td><td>2</td><td>Studio: LFM3-Omni audio input, export, LoRA defaults and Stop for custom-code generate</td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/issues/8502">deepspeedai/DeepSpeed#8502</a></td><td>2</td><td>Restore full training state in checkpoint writer engines</td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/issues/8745">deepspeedai/DeepSpeed#8745</a></td><td>2</td><td>AutoEP: opt-in skip of the recomputed DeepEP combine</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47830">sgl-project/sglang#47830</a></td><td>2</td><td>[Bug] Hybrid Mamba model + EAGLE: KV-cache sizing error says to raise --mem-fraction-static when the Mamba state cache is what fills memory</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47770">sgl-project/sglang#47770</a></td><td>2</td><td>[Perf] DFlash: Use CUDA graphs for draft and idle verify under DP attention</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47729">sgl-project/sglang#47729</a></td><td>2</td><td>[Feat] qwen3.5: Support native sequence classification</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/42551">sgl-project/sglang#42551</a></td><td>2</td><td>[Feature] Support API keys from secret environment variables</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/42550">sgl-project/sglang#42550</a></td><td>2</td><td>[Fix] Reject removal of an external corpus while its load is pending</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/42549">sgl-project/sglang#42549</a></td><td>2</td><td>[Fix] Preserve Gemma4 null and exponent tool argument types</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/37349">sgl-project/sglang#37349</a></td><td>2</td><td>[Guide] Setup and Troubleshooting for SGLang on V100</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/43102">sgl-project/sglang#43102</a></td><td>2</td><td>[Feat] [NPU] qwen4: Support Qwen3.8-Flash-Next W8A8 inference with QSA on Ascend A2</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/40356">sgl-project/sglang#40356</a></td><td>2</td><td>[Fix] [Ascend] bench_serving: Correct Kimi K2.5 image token counts</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47416">sgl-project/sglang#47416</a></td><td>2</td><td>[Fix] metrics: Correct MFU/MBU bandwidth overcounting and GraniteMoE AutoTokenizer fallback</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/37052">sgl-project/sglang#37052</a></td><td>2</td><td>[Bug] Qwen3.8-Flash-Next + NEXTN full decode graph: repeated dual-rank invalid-probability asserts on GB10</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/37097">sgl-project/sglang#37097</a></td><td>2</td><td>[Bug] Pretokenized image requests can crash GLM MRoPE with a stale retokenized mask</td></tr><tr><td><a href="https://github.com/huggingface/diffusers/issues/15161">huggingface/diffusers#15161</a></td><td>2</td><td>DreamLite: disabling attention slicing replaces the GQA processor and breaks inference</td></tr><tr><td><a href="https://github.com/huggingface/lerobot/issues/5099">huggingface/lerobot#5099</a></td><td>2</td><td>[Discussion] Has the bf16 update-erasure paper been evaluated or reproduced in LeRobot?</td></tr><tr><td><a href="https://github.com/verl-project/verl/issues/8414">verl-project/verl#8414</a></td><td>2</td><td>[model] perf: build Qwen3-VL/Qwen3.5 pos-embed interpolation tensors without Python lists</td></tr><tr><td><a href="https://github.com/huggingface/peft/issues/4086">huggingface/peft#4086</a></td><td>2</td><td>Fix broken examples on transformers 5.x (target_modules, dataset IDs, warmup_ratio)</td></tr><tr><td><a href="https://github.com/huggingface/peft/issues/3938">huggingface/peft#3938</a></td><td>2</td><td>FIX Reshard on disable_adapter() exit only when needed</td></tr><tr><td><a href="https://github.com/huggingface/trl/issues/8006">huggingface/trl#8006</a></td><td>2</td><td>Fail CI test jobs on self-hosted runners when pytest does not complete</td></tr><tr><td><a href="https://github.com/huggingface/trl/issues/8058">huggingface/trl#8058</a></td><td>2</td><td>[WIP] Quantize the tiny DeepSeek-V3 and GPT-OSS models like their references (FP8, MXFP4)</td></tr><tr><td><a href="https://github.com/huggingface/trl/issues/8050">huggingface/trl#8050</a></td><td>2</td><td>`parse_response` puts Qwen3.5 tool calls into `reasoning_content` when the model skips the prefilled `&lt;think&gt;`</td></tr><tr><td><a href="https://github.com/huggingface/trl/issues/7965">huggingface/trl#7965</a></td><td>2</td><td>[BUG] GRPO/RLOO continuous batching casts the trained weights to bf16 in place</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4582">NVIDIA-NeMo/Automodel#4582</a></td><td>2</td><td>fix(minimax_m3_vl): capture DSpark last-layer features before the final norm</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4584">NVIDIA-NeMo/Automodel#4584</a></td><td>2</td><td>fix(minimax_m3_vl): accept SM103 (B300/GB300) for the MSA kernels</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4532">NVIDIA-NeMo/Automodel#4532</a></td><td>2</td><td>feat(moe): support Dist-MoE expert backend </td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4551">NVIDIA-NeMo/Automodel#4551</a></td><td>2</td><td>feat(data): add a common finite-stream loader contract</td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/issues/9726">modelscope/ms-swift#9726</a></td><td>2</td><td>fix: Call destroy_process_group() at the end of the trainer pipelines</td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/issues/7875">modelscope/ms-swift#7875</a></td><td>2</td><td>自己写的pt和vllm推理差异巨大 原框架的pt和vllm也有差异</td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/issues/8164">modelscope/ms-swift#8164</a></td><td>2</td><td>Bidirectional mask for embedding training w/ megatron</td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/issues/10307">modelscope/ms-swift#10307</a></td><td>2</td><td>autoawq 和gptqmodel依赖与torch版本冲突</td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/issues/8329">modelscope/ms-swift#8329</a></td><td>2</td><td>ms-swift4.0.1+transformer 5.2.0微调qwen3-vl-30b-a3b,部署量化模型时IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)</td></tr><tr><td><a href="https://github.com/EleutherAI/lm-evaluation-harness/issues/4440">EleutherAI/lm-evaluation-harness#4440</a></td><td>2</td><td>Fix xquad, coqa, record and other QA tasks on transformers 5.20 (squad_metrics was removed)</td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/issues/4119">axolotl-ai-cloud/axolotl#4119</a></td><td>2</td><td>chat_template: text-diff turn locator trains the role-header token when content starts with whitespace (Phi-3.5, Phi-4)</td></tr><tr><td><a href="https://github.com/huggingface/accelerate/issues/4359">huggingface/accelerate#4359</a></td><td>2</td><td>`gather_for_metrics` keeps padding duplicates for a sized `IterableDataset` with `dispatch_batches=False`</td></tr><tr><td><a href="https://github.com/THUDM/slime/issues/1810">THUDM/slime#1810</a></td><td>2</td><td>[Question] 多轮sft/多轮function call sft数据集组织形式</td></tr><tr><td><a href="https://github.com/vllm-project/llm-compressor/issues/3420">vllm-project/llm-compressor#3420</a></td><td>2</td><td>add GLM-MoE-DSA MTP support</td></tr><tr><td><a href="https://github.com/allenai/open-instruct/issues/2141">allenai/open-instruct#2141</a></td><td>2</td><td>Add named MILES candidate sources and verifier qualification</td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/issues/3480">ModelCloud/GPTQModel#3480</a></td><td>2</td><td>fix(cpu): new cpu kernels regressed trainable api calls from peft</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2278">unslothai/unsloth-zoo#2278</a></td><td>2</td><td>Merge: apply a trust_remote_code model&#x27;s registered checkpoint renames, and refuse a merge that places no LoRA</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/65075">vllm-project/vllm#65075</a></td><td>1</td><td>[Bug]: Bi-encoder scoring token limits fail with MistralTokenizer</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/65074">vllm-project/vllm#65074</a></td><td>1</td><td>[Bug]: MCP tool discovery drops tools after the first tools/list page</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/65081">vllm-project/vllm#65081</a></td><td>1</td><td>[CI Failure]: pre-commit mypy on main - process_weights_after_loading redefined in vllm/models/muse_spark/amd/model.py</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/65071">vllm-project/vllm#65071</a></td><td>1</td><td>[Bug]: CustomDataset converts numeric-string prompts to integers before tokenization</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/65072">vllm-project/vllm#65072</a></td><td>1</td><td>[Bug]: ASRDataset uses channel count as duration when filtering stereo audio bytes</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/63906">vllm-project/vllm#63906</a></td><td>1</td><td>[Bug]: FunctionGemma converts escape-delimited string literals into numbers, booleans and null</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/65048">vllm-project/vllm#65048</a></td><td>1</td><td>[CI Failure]: MI300 MoE Kernels Multi-GPU - MoriEPv2 combine block_num 304 exceeds xdb_flag_slots 256 in test_modular_kernel_combinations.py</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/50792">vllm-project/vllm#50792</a></td><td>1</td><td>[Bug]: OverflowError crash in `prompt_logprobs` decode path on PD-disaggregated (external KV connector) setup</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/65035">vllm-project/vllm#65035</a></td><td>1</td><td>[Bug]: Anthropic Messages stream reports the whole prompt as input_tokens in message_start</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/63487">vllm-project/vllm#63487</a></td><td>1</td><td>[Bugfix] Propagate EngineDeadError on AsyncLLM output_handler cancellation</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/65015">vllm-project/vllm#65015</a></td><td>1</td><td>[Bugfix][Tool Parser] Keep granite4 streamed text between tool calls</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/65002">vllm-project/vllm#65002</a></td><td>1</td><td>[Model] Add NVIDIA Nemotron-3-Diarization (offline speaker diarization)</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64809">vllm-project/vllm#64809</a></td><td>1</td><td>[Bug]: `raw_logprobs` are computed after the structured-output grammar mask</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64956">vllm-project/vllm#64956</a></td><td>1</td><td>[Bug]: kv_cache_dtype=&quot;turboquant_3bit_nc&quot; crashes on Phi-2 (head_size=80) — supports_head_size() claims &quot;any positive value&quot;, but the Hadamard rotation is built at the next power of two (128×128) and matmul&#x27;d with an 80-dim key</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64961">vllm-project/vllm#64961</a></td><td>1</td><td>[Bugfix][Frontend] Convert Responses content parts to chat parts</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64953">vllm-project/vllm#64953</a></td><td>1</td><td>[Bug]: `seed=-1` passes config validation and kills engine init inside `np.random.seed` (&quot;Seed must be between 0 and 2**32 - 1&quot;) — while `SamplingParams` already treats `-1` as &quot;no seed&quot;</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64885">vllm-project/vllm#64885</a></td><td>1</td><td>[Bugfix][Pooling] Enable Matryoshka dimensions for EmbeddingGemma 2</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64903">vllm-project/vllm#64903</a></td><td>1</td><td>[Bugfix][MoE] Avoid double router weighting in TRTLLM FP8 experts</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64845">vllm-project/vllm#64845</a></td><td>1</td><td>[Bug]: EmbeddingGemma 2 rejects `dimensions` by default although the model supports Matryoshka (128/256/512/768)</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16407">unslothai/unsloth#16407</a></td><td>1</td><td>UNSLOTH_PARALLEL: one flag and one root mesh for dp / tp / pp / cp, pipeline parallel from the environment, and a capability registry</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16416">unslothai/unsloth#16416</a></td><td>1</td><td>Packed mRoPE attention: no graph break on warm packed rows</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16169">unslothai/unsloth#16169</a></td><td>1</td><td>[Bug] Sandbox error that ejects the model?</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16406">unslothai/unsloth#16406</a></td><td>1</td><td>Pipeline parallel: carry the Qwen3-MoE router auxiliary loss across stages</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16410">unslothai/unsloth#16410</a></td><td>1</td><td>Block compile: install the router_logits capture hooks before the first block compiles</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16331">unslothai/unsloth#16331</a></td><td>1</td><td>CLI: no way to pass trust_remote_code, so models with custom code cannot load</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16332">unslothai/unsloth#16332</a></td><td>1</td><td>CLI: let train load models that need trust_remote_code</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16221">unslothai/unsloth#16221</a></td><td>1</td><td>[Bug] ShareGPT image lists fail after passing dataset format checks</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16294">unslothai/unsloth#16294</a></td><td>1</td><td>[Bug] Studio video: MiniMax-H3 on Auto attention crashes with &quot;No available kernel&quot; on Windows (torch without flash attention)</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/15632">unslothai/unsloth#15632</a></td><td>1</td><td>Block Swap blocker: Qwen3.8-Flash-Next-FP8 can&#x27;t train with offload_layers, its FP8 n-gram table loads without its weight_scale</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16252">unslothai/unsloth#16252</a></td><td>1</td><td>Tests: compare the disabled 4-bit fused LoRA forward with dequantize + linear, not bitsandbytes&#x27; fused GEMM</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16104">unslothai/unsloth#16104</a></td><td>1</td><td>Saving: write the source tokenizer_class instead of TokenizersBackend</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16102">unslothai/unsloth#16102</a></td><td>1</td><td>Saving: drop the training truncation and padding from saved tokenizers</td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/issues/10868">hiyouga/LlamaFactory#10868</a></td><td>1</td><td>enable fp16 in dtype inference on Intel XPU</td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/issues/8950">deepspeedai/DeepSpeed#8950</a></td><td>1</td><td>Fix torch-latest workflow unit failures</td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/issues/8851">deepspeedai/DeepSpeed#8851</a></td><td>1</td><td>[AutoTP] Support frozen AutoTP models and manage AutoTP mode per model</td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/issues/8843">deepspeedai/DeepSpeed#8843</a></td><td>1</td><td>Keep the parameters a model names for fp32 in fp32 under ZeRO-3</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/46303">sgl-project/sglang#46303</a></td><td>1</td><td>[Perf] diffusion: fix 6 quantization-layer overheads (NVFP4 scale conversion, W8A16, BnB, Nunchaku)</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/46248">sgl-project/sglang#46248</a></td><td>1</td><td>[Feat] hicache: Add unified Engram store and support Qwen4-Exp</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47841">sgl-project/sglang#47841</a></td><td>1</td><td>[Fix] transcription: Serve /v1/audio/transcriptions for Qwen3-Omni and Gemma 4</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/45798">sgl-project/sglang#45798</a></td><td>1</td><td>[Fix] llava: Load LLaVA-1.5 CLIP tower</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/39972">sgl-project/sglang#39972</a></td><td>1</td><td>[Model] Add BerryLM</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/37648">sgl-project/sglang#37648</a></td><td>1</td><td>[Bug][ROCm] GLM-5.3-MXFP4 TP8 + EAGLE GPU memory fault under high-concurrency long-context load</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/37646">sgl-project/sglang#37646</a></td><td>1</td><td>[Bug] --enable-flashinfer-pure-allreduce is not available on 0.5.18</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/37645">sgl-project/sglang#37645</a></td><td>1</td><td>[Bug] glm45 reasoning parser returns the whole answer as reasoning_content (content empty) when the model skips thinking and stops on EOS</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47786">sgl-project/sglang#47786</a></td><td>1</td><td>[Feat] multimodal: Share CPU preprocessing artifacts across tokenizer workers</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47060">sgl-project/sglang#47060</a></td><td>1</td><td>[Fix] observability: Snapshot STANDALONE metrics before settlement</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47746">sgl-project/sglang#47746</a></td><td>1</td><td>[Fix] diffusion: SenseNova-U1: keep think tokens in attention, fix 4.x import and vision_model init</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47649">sgl-project/sglang#47649</a></td><td>1</td><td>[Fix] scoring: Return token_logprobs for setwise CausalLM scoring</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47592">sgl-project/sglang#47592</a></td><td>1</td><td>[CI] ci: Preload `sglang.test.test_utils` in CPU fork workers</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/37283">sgl-project/sglang#37283</a></td><td>1</td><td>[Bug] A non-object entry in a tool-call array discards the valid calls beside it</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/37215">sgl-project/sglang#37215</a></td><td>1</td><td>[Bug] --dp 8 intermittently fails with TCPStore EADDRINUSE on single-node 8×H800</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/37186">sgl-project/sglang#37186</a></td><td>1</td><td>[Bug] Qwen3.5 visual encoder weights dropped by PP layer-range filter in load_weights</td></tr><tr><td><a href="https://github.com/huggingface/diffusers/issues/15121">huggingface/diffusers#15121</a></td><td>1</td><td>Fix several quantizer config bugs (AutoRound override, ModelOpt JSON crash, single-file pre_quantized, config mutation, Nunchaku group_size, dataclass eq/repr)</td></tr><tr><td><a href="https://github.com/huggingface/diffusers/issues/15163">huggingface/diffusers#15163</a></td><td>1</td><td>Flux and LTX pipelines do not forward the generator to stochastic scheduler steps</td></tr><tr><td><a href="https://github.com/huggingface/lerobot/issues/4775">huggingface/lerobot#4775</a></td><td>1</td><td>feat(policies): load pi0, pi0.5 and pi0-FAST checkpoints without first building random weights</td></tr><tr><td><a href="https://github.com/huggingface/lerobot/issues/5146">huggingface/lerobot#5146</a></td><td>1</td><td>fix(eo1): give the slow Qwen2.5-VL image processor CPU images</td></tr><tr><td><a href="https://github.com/huggingface/lerobot/issues/5162">huggingface/lerobot#5162</a></td><td>1</td><td>fix(policies): let EO-1, WALL-X and Wall-OSS 0.5 pass torch.export</td></tr><tr><td><a href="https://github.com/huggingface/lerobot/issues/5134">huggingface/lerobot#5134</a></td><td>1</td><td>feat(policies): add Cosmos3</td></tr><tr><td><a href="https://github.com/verl-project/verl/issues/8404">verl-project/verl#8404</a></td><td>1</td><td>[hardware, megatron, doc] feat: upgrade Ascend 950 to Megatron-Core 0.18 and add MXFP8 GRPO example</td></tr><tr><td><a href="https://github.com/huggingface/peft/issues/4096">huggingface/peft#4096</a></td><td>1</td><td>Update FSDP docs</td></tr><tr><td><a href="https://github.com/huggingface/peft/issues/4094">huggingface/peft#4094</a></td><td>1</td><td>reduce_intruder_dimension gives wrong LoRA factors for fan_in_fan_out (Conv1D) and Embedding layers</td></tr><tr><td><a href="https://github.com/huggingface/peft/issues/4090">huggingface/peft#4090</a></td><td>1</td><td>layers_to_transform without layers_pattern matches nothing when the layer list is at the model root</td></tr><tr><td><a href="https://github.com/huggingface/peft/issues/4088">huggingface/peft#4088</a></td><td>1</td><td>[v1 #4071] Disjoint auxiliary trainable tokens activate only the first adapter</td></tr><tr><td><a href="https://github.com/huggingface/peft/issues/4093">huggingface/peft#4093</a></td><td>1</td><td>CI: Bump the third-party-actions group with 5 updates</td></tr><tr><td><a href="https://github.com/huggingface/trl/issues/8040">huggingface/trl#8040</a></td><td>1</td><td>Add experimental DecisionTrainer</td></tr><tr><td><a href="https://github.com/huggingface/trl/issues/8054">huggingface/trl#8054</a></td><td>1</td><td>Allow `&lt;tool_call&gt;` to close reasoning region in `qwen3_5_template`</td></tr><tr><td><a href="https://github.com/huggingface/trl/issues/8052">huggingface/trl#8052</a></td><td>1</td><td>SFTTrainer silently skips ModernBERT Decoder&#x27;s prediction head</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4515">NVIDIA-NeMo/Automodel#4515</a></td><td>1</td><td>feat(speculative): add MTP accept-length eval and finetune example</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4555">NVIDIA-NeMo/Automodel#4555</a></td><td>1</td><td>feat(data): integrate Zephon image training across VLM families</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4556">NVIDIA-NeMo/Automodel#4556</a></td><td>1</td><td>test(zephon): cover distributed replay and model parity</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4554">NVIDIA-NeMo/Automodel#4554</a></td><td>1</td><td>refactor(data): enable sharing VLM preprocessing across data load</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4553">NVIDIA-NeMo/Automodel#4553</a></td><td>1</td><td>feat(data): add Zephon instruction tuning</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4552">NVIDIA-NeMo/Automodel#4552</a></td><td>1</td><td>feat: support Zephon for text pretraining</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4550">NVIDIA-NeMo/Automodel#4550</a></td><td>1</td><td>fix(packing): preserve native packed attention through PP</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4549">NVIDIA-NeMo/Automodel#4549</a></td><td>1</td><td>fix(pp): preserve native model media ownership</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4548">NVIDIA-NeMo/Automodel#4548</a></td><td>1</td><td>feat(data): add shared packed attention and media handoff</td></tr><tr><td><a href="https://github.com/EleutherAI/lm-evaluation-harness/issues/4369">EleutherAI/lm-evaluation-harness#4369</a></td><td>1</td><td>Fix greedy token matching in the SGLang generate API</td></tr><tr><td><a href="https://github.com/huggingface/accelerate/issues/4319">huggingface/accelerate#4319</a></td><td>1</td><td>Fix disabled ZeRO-3 initialization contexts</td></tr><tr><td><a href="https://github.com/huggingface/accelerate/issues/4475">huggingface/accelerate#4475</a></td><td>1</td><td>FSDP2 cpu_ram_efficient_loading: honor ignored modules and load buffers</td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/issues/1579">linkedin/Liger-Kernel#1579</a></td><td>1</td><td>[Feature] Support GLM-5.3-Flash (glm5_next): fused linear cross-entropy and RMSNorm</td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/issues/1447">linkedin/Liger-Kernel#1447</a></td><td>1</td><td>[WIP] Add Qwen4Exp text model support</td></tr><tr><td><a href="https://github.com/Blaizzy/mlx-vlm/issues/2640">Blaizzy/mlx-vlm#2640</a></td><td>1</td><td>Make the LFM3-VL processor follow the transformers processor contract</td></tr><tr><td><a href="https://github.com/Blaizzy/mlx-vlm/issues/2639">Blaizzy/mlx-vlm#2639</a></td><td>1</td><td>LFM3-VL processor does not follow the transformers processor contract</td></tr><tr><td><a href="https://github.com/Blaizzy/mlx-vlm/issues/2638">Blaizzy/mlx-vlm#2638</a></td><td>1</td><td>20261031 -- Diagnostic update: 45-model sweep on 0.7.11 @ 4693cd0e: no crashes, nothing new; LFM3-VL and Qwen3.6 run cleanly</td></tr><tr><td><a href="https://github.com/EvolvingLMMs-Lab/lmms-eval/issues/1625">EvolvingLMMs-Lab/lmms-eval#1625</a></td><td>1</td><td>[Task] RoboAfford-Eval</td></tr><tr><td><a href="https://github.com/vllm-project/llm-compressor/issues/3427">vllm-project/llm-compressor#3427</a></td><td>1</td><td>[Bug]: NVFP4 MoE example (qwen_30b_a3b.py) gives unstable, damaging expert activation scales: unseeded 20-sample calibration</td></tr><tr><td><a href="https://github.com/allenai/open-instruct/issues/2108">allenai/open-instruct#2108</a></td><td>1</td><td>fix: restore FP32 RoPE frequencies after DeepSpeed BF16 initialization</td></tr><tr><td><a href="https://github.com/huggingface/optimum/issues/2455">huggingface/optimum#2455</a></td><td>1</td><td>fix #2432: support transformers&gt;=5.0.0 and fix torch.load security warning</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2360">unslothai/unsloth-zoo#2360</a></td><td>1</td><td>Gemma-4 forced float32: keep norms float32 on full finetuning so lm_head gets float32</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2352">unslothai/unsloth-zoo#2352</a></td><td>1</td><td>Compiler: keep softmax outputs float32 when the source asks for float32</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2350">unslothai/unsloth-zoo#2350</a></td><td>1</td><td>Granite-4 hybrid: cast float32-weight norm outputs back to 16-bit without autocast</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2343">unslothai/unsloth-zoo#2343</a></td><td>1</td><td>Save Llama-4 and Qwen3-VL-MoE expert LoRA in the layout vanilla PEFT loads</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2348">unslothai/unsloth-zoo#2348</a></td><td>1</td><td>prepare_model_for_training: cast float32 bias masters at every owner of a tied bias</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2345">unslothai/unsloth-zoo#2345</a></td><td>1</td><td>MoE LoRA stash probe: measure the experts forward, not the gpt-oss decode route</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2344">unslothai/unsloth-zoo#2344</a></td><td>1</td><td>Qwen3-MoE forced float32: fix full finetuning on GPUs without bfloat16</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2323">unslothai/unsloth-zoo#2323</a></td><td>1</td><td>Answer the transposed-view check together with the home grouped_mm probe</td></tr><tr><td><a href="https://github.com/ggml-org/llama.cpp/issues/30922">ggml-org/llama.cpp#30922</a></td><td>0</td><td>mtmd : resize Gemma 4 images without letterbox padding</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/65073">vllm-project/vllm#65073</a></td><td>0</td><td>[Bug]: Duplicate MCP server names advertise tools from a different execution server</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/63300">vllm-project/vllm#63300</a></td><td>0</td><td>[Rust Frontend] Complete partial cached HF snapshots when online</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/65055">vllm-project/vllm#65055</a></td><td>0</td><td>[Performance]: MoE top-k gating PDL launch (#59737) slows CUDA graph replay on pre-Hopper GPUs</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/63983">vllm-project/vllm#63983</a></td><td>0</td><td>[Bug]: FunctionGemma tool parser drops every non-string argument (numbers, booleans, arrays, objects)</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64541">vllm-project/vllm#64541</a></td><td>0</td><td>[Bug]: dots.ocr produces identical &quot;blurry image&quot; output for every region with tensor-parallel size 2</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64418">vllm-project/vllm#64418</a></td><td>0</td><td>[RFC]: Load plan — an engine-neutral API for sharded weight transfer</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/64954">vllm-project/vllm#64954</a></td><td>0</td><td>[Bug]: `--attention-backend TORCH_SDPA` passes config validation, then dies with &quot;Backend TORCH_SDPA must be registered before use&quot; — a ViT-only tag offered as a valid option, whose `ValueError` bypasses the &quot;Selected backend ... is not valid&quot; path</td></tr><tr><td><a href="https://github.com/vllm-project/vllm/issues/62470">vllm-project/vllm#62470</a></td><td>0</td><td>[Bugfix][LoRA] Scale embedding LoRA deltas by the base layer&#x27;s embed_scale</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16413">unslothai/unsloth#16413</a></td><td>0</td><td>Studio: keep a generated chat title in the conversation&#x27;s language</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16377">unslothai/unsloth#16377</a></td><td>0</td><td>Studio: apply the mapped chat template once per loaded tokenizer</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/15765">unslothai/unsloth#15765</a></td><td>0</td><td>Studio: show measured transcription progress</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/16279">unslothai/unsloth#16279</a></td><td>0</td><td>Ascend: opt-in eager bootstrap for Torch 2.6/Triton 3.5 on ARM64</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/15633">unslothai/unsloth#15633</a></td><td>0</td><td>Block Swap blocker: offload_layers never engages on Qwen3.8-Flash-Next, the CPU-kept n-gram table makes the map look like two devices</td></tr><tr><td><a href="https://github.com/unslothai/unsloth/issues/15159">unslothai/unsloth#15159</a></td><td>0</td><td>Block Swap: get_peft_model crashes on MoE models with offload_layers (evicted expert weights read as empty)</td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/issues/10929">hiyouga/LlamaFactory#10929</a></td><td>0</td><td>[data] Preserve literal placeholders in user content during template encoding</td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/issues/10876">hiyouga/LlamaFactory#10876</a></td><td>0</td><td>[xpu] support profiler, module profiler and env report on Intel GPU</td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/issues/10927">hiyouga/LlamaFactory#10927</a></td><td>0</td><td>[chat] pass mm_token_type_ids to generate for mrope models</td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/issues/10926">hiyouga/LlamaFactory#10926</a></td><td>0</td><td>[data] avoid mutating Qwen video frame inputs</td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/issues/10924">hiyouga/LlamaFactory#10924</a></td><td>0</td><td>feat(data): optionally drop overlong SFT examples</td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/issues/10923">hiyouga/LlamaFactory#10923</a></td><td>0</td><td>feat(webui): support private Hub export repositories</td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/issues/10849">hiyouga/LlamaFactory#10849</a></td><td>0</td><td>[data] guard Qwen2-VL media tokens against SFT truncation</td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/issues/10918">hiyouga/LlamaFactory#10918</a></td><td>0</td><td>[npu] Skip unnecessary gradients in NPU fused MoE backward</td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/issues/10922">hiyouga/LlamaFactory#10922</a></td><td>0</td><td>fix: handle a single validation sample after CLI parsing</td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/issues/8932">deepspeedai/DeepSpeed#8932</a></td><td>0</td><td>[REQUEST][Hybrid Engine] Enable graph capture for HybridEngine on non-CUDA accelerators</td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/issues/8962">deepspeedai/DeepSpeed#8962</a></td><td>0</td><td>Fix AutoTP multi-query GPTBigCode training at TP &gt; 1</td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/issues/8961">deepspeedai/DeepSpeed#8961</a></td><td>0</td><td>[BUG] AutoTP multi-query GPTBigCode training fails at TP &gt; 1</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/43864">sgl-project/sglang#43864</a></td><td>0</td><td>[Bug] gemma-4-31B-it: audio request crashes the scheduler; /get_model_info misreports has_audio_understanding: true</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47731">sgl-project/sglang#47731</a></td><td>0</td><td>[Perf] imports: Keep light modules off HF patches, model configs, and the scheduler</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47710">sgl-project/sglang#47710</a></td><td>0</td><td>[Bug] SM120: fused-A GEMM eligibility accepts K % 256 but the auto-selected CuTe DSL kernel asserts K % 1024 (hidden_size=3584 crashes)</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47809">sgl-project/sglang#47809</a></td><td>0</td><td>[Fix] attention: Honor custom masks past the causal tile in Triton extend</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47799">sgl-project/sglang#47799</a></td><td>0</td><td>[Fix] models: Phi-3-4k ignores its 2047-token sliding window (output breaks past 2k tokens)</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47759">sgl-project/sglang#47759</a></td><td>0</td><td>[Bug] Video input data still leak into logs at --log-requests-level 0/1</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/41274">sgl-project/sglang#41274</a></td><td>0</td><td>[sglang-miles] Recover disk-delta checkpoints safely across resets and retries</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47701">sgl-project/sglang#47701</a></td><td>0</td><td>[Feat] activevln: Add differentiable actor and SRT weight export</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47444">sgl-project/sglang#47444</a></td><td>0</td><td>[Bug] NemotronHConfig rejects transformers 5 layer types `linear_attention` / `full_attention` in config.json</td></tr><tr><td><a href="https://github.com/sgl-project/sglang/issues/47223">sgl-project/sglang#47223</a></td><td>0</td><td>[Fix] constrained: Share one compile among concurrent requests for a grammar key</td></tr><tr><td><a href="https://github.com/huggingface/diffusers/issues/15168">huggingface/diffusers#15168</a></td><td>0</td><td>Bump the actions group across 1 directory with 16 updates</td></tr><tr><td><a href="https://github.com/huggingface/diffusers/issues/15167">huggingface/diffusers#15167</a></td><td>0</td><td>UniPC and DPM-Solver with use_flow_sigmas condition the model on integer timesteps while integrating with float sigmas</td></tr><tr><td><a href="https://github.com/huggingface/diffusers/issues/15166">huggingface/diffusers#15166</a></td><td>0</td><td>[training scripts] `--logit_mean` biases timesteps in the opposite direction to the SD3 paper (positive m oversamples low noise)</td></tr><tr><td><a href="https://github.com/huggingface/diffusers/issues/15164">huggingface/diffusers#15164</a></td><td>0</td><td>Add Kandinsky S2V 1.0 (speech-to-video) as a modular pipeline</td></tr><tr><td><a href="https://github.com/huggingface/diffusers/issues/15145">huggingface/diffusers#15145</a></td><td>0</td><td>flux_lora_quantization example: compute_embeddings.py crashes on current transformers</td></tr><tr><td><a href="https://github.com/huggingface/lerobot/issues/5194">huggingface/lerobot#5194</a></td><td>0</td><td>feat(sim): NVIDIA RoboLab benchmark on Isaac Lab</td></tr><tr><td><a href="https://github.com/huggingface/lerobot/issues/5189">huggingface/lerobot#5189</a></td><td>0</td><td>`lerobot-train` profile on one H200 (ACT, SmolVLA, Diffusion): where step time and memory go</td></tr><tr><td><a href="https://github.com/huggingface/lerobot/issues/5188">huggingface/lerobot#5188</a></td><td>0</td><td>perf(pi0): share one read-only prefix cache across the denoise steps</td></tr><tr><td><a href="https://github.com/huggingface/lerobot/issues/5067">huggingface/lerobot#5067</a></td><td>0</td><td>perf(smolvla): precompute cross-attention expert K/V once per chunk</td></tr><tr><td><a href="https://github.com/verl-project/verl/issues/8097">verl-project/verl#8097</a></td><td>0</td><td>[fsdp, model] fix: gather DTensor LM heads for fused kernels</td></tr><tr><td><a href="https://github.com/huggingface/peft/issues/4085">huggingface/peft#4085</a></td><td>0</td><td>Fix modules_to_save selection when merging an inactive adapter</td></tr><tr><td><a href="https://github.com/huggingface/peft/issues/4089">huggingface/peft#4089</a></td><td>0</td><td>FEAT healthcheck: warn when a randomly initialized task head is not saved</td></tr><tr><td><a href="https://github.com/huggingface/trl/issues/7960">huggingface/trl#7960</a></td><td>0</td><td>Tool calling is not supported for some popular model families</td></tr><tr><td><a href="https://github.com/huggingface/trl/issues/8056">huggingface/trl#8056</a></td><td>0</td><td>GRPO/RLOO under tensor parallelism: TP ranks sample different completions, and the prompt sampler is sized for every process</td></tr><tr><td><a href="https://github.com/huggingface/trl/issues/8051">huggingface/trl#8051</a></td><td>0</td><td>Docs: smol-course links in the SFT, DPO and ORPO trainer pages return 404</td></tr><tr><td><a href="https://github.com/huggingface/sentence-transformers/issues/4311">huggingface/sentence-transformers#4311</a></td><td>0</td><td>[fix] Let MSEEvaluator work without a teacher_model</td></tr><tr><td><a href="https://github.com/huggingface/sentence-transformers/issues/4310">huggingface/sentence-transformers#4310</a></td><td>0</td><td>[fix] Stop RerankingEvaluator crashes on single-document samples and empty sample sets</td></tr><tr><td><a href="https://github.com/huggingface/sentence-transformers/issues/4309">huggingface/sentence-transformers#4309</a></td><td>0</td><td>Fix proportional sampling dropping batches beyond estimated child lengths</td></tr><tr><td><a href="https://github.com/huggingface/sentence-transformers/issues/4308">huggingface/sentence-transformers#4308</a></td><td>0</td><td>Reduce CPU XTR training-scorer memory with streaming token top-k</td></tr><tr><td><a href="https://github.com/huggingface/sentence-transformers/issues/4307">huggingface/sentence-transformers#4307</a></td><td>0</td><td>Support variable-length hard negatives in CrossEncoder ranking losses</td></tr><tr><td><a href="https://github.com/huggingface/sentence-transformers/issues/4306">huggingface/sentence-transformers#4306</a></td><td>0</td><td>[fix] Make SparseAutoEncoder picklable so CSR models work with multi-process encoding and DataLoader workers</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4578">NVIDIA-NeMo/Automodel#4578</a></td><td>0</td><td>MiMo V2 overlength prefilter estimates image tokens with the HF resize helper, so samples over `max_length` are kept and then never truncated</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/issues/4557">NVIDIA-NeMo/Automodel#4557</a></td><td>0</td><td>`tools/merge_lora.py`: merging an `lm_head` LoRA adapter into a model with `tie_word_embeddings=true` silently changes the input embedding table</td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/issues/10492">modelscope/ms-swift#10492</a></td><td>0</td><td>fix(dataset): keep message/tool columns intact when map writes run in spawned workers</td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/issues/10491">modelscope/ms-swift#10491</a></td><td>0</td><td>Add GPTQModel-backed AWQ export</td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/issues/10489">modelscope/ms-swift#10489</a></td><td>0</td><td>[megatron] Wrap DDP/Megatron-FSDP on the current stream in eager mode</td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/issues/10487">modelscope/ms-swift#10487</a></td><td>0</td><td>fix(trainers): avoid gradient accumulation scaling during evaluation</td></tr><tr><td><a href="https://github.com/FlagOpen/FlagEmbedding/issues/1622">FlagOpen/FlagEmbedding#1622</a></td><td>0</td><td>fix: preserve dense pooling in encoder checkpoints</td></tr><tr><td><a href="https://github.com/FlagOpen/FlagEmbedding/issues/1621">FlagOpen/FlagEmbedding#1621</a></td><td>0</td><td>fix: load auto models from paths with trailing separators</td></tr><tr><td><a href="https://github.com/FlagOpen/FlagEmbedding/issues/1620">FlagOpen/FlagEmbedding#1620</a></td><td>0</td><td>fix: require explicit opt-in for embedder remote code</td></tr><tr><td><a href="https://github.com/FlagOpen/FlagEmbedding/issues/1619">FlagOpen/FlagEmbedding#1619</a></td><td>0</td><td>fix: preserve layerwise scores across reranker worker batches</td></tr><tr><td><a href="https://github.com/huggingface/accelerate/issues/4448">huggingface/accelerate#4448</a></td><td>0</td><td>Speed up forward with offloaded weights</td></tr><tr><td><a href="https://github.com/bitsandbytes-foundation/bitsandbytes/issues/2116">bitsandbytes-foundation/bitsandbytes#2116</a></td><td>0</td><td>LAMB / LARS never update a parameter whose norm is zero (zero-initialised biases, LoRA B matrices, zero-init residual gains)</td></tr><tr><td><a href="https://github.com/ml-explore/mlx-lm/issues/2026">ml-explore/mlx-lm#2026</a></td><td>0</td><td>perf(gemma4): add per_layer_config, KV-shared logits_to_keep prefill trimming, and MTP/MoE sanitization</td></tr><tr><td><a href="https://github.com/ml-explore/mlx-lm/issues/2009">ml-explore/mlx-lm#2009</a></td><td>0</td><td>fix(gpt_oss): detect transformers [E, H, 2F] layout in sanitize() (fixes #1994)</td></tr><tr><td><a href="https://github.com/ml-explore/mlx-lm/issues/2025">ml-explore/mlx-lm#2025</a></td><td>0</td><td>Support transformers_weights in config.json</td></tr><tr><td><a href="https://github.com/ml-explore/mlx-lm/issues/2024">ml-explore/mlx-lm#2024</a></td><td>0</td><td>Fix hardcoded GELU in &quot;gpt_neox&quot; and &quot;gpt_bigcode&quot;</td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/issues/1391">linkedin/Liger-Kernel#1391</a></td><td>0</td><td>Add FP32 accumulation option to fused scaled cross entropy</td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/issues/1571">linkedin/Liger-Kernel#1571</a></td><td>0</td><td>CuTe DSL RoPE returns wrong results after the first call when CUTE_DSL_ENABLE_TVM_FFI=1</td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/issues/1581">linkedin/Liger-Kernel#1581</a></td><td>0</td><td>fix(cutedsl): use safe low-overhead RoPE launch bindings</td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/issues/1587">linkedin/Liger-Kernel#1587</a></td><td>0</td><td>Restore torch cross-entropy state in revert_liger_kernel helpers</td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/issues/1574">linkedin/Liger-Kernel#1574</a></td><td>0</td><td>fix(cross_entropy): reduce per-token loss in fp32 and scale reduction=&quot;none&quot; gradients in place</td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/issues/1583">linkedin/Liger-Kernel#1583</a></td><td>0</td><td>fix(grpo): keep the Triton gradient when delta &lt; 1 + epsilon_high</td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/issues/1582">linkedin/Liger-Kernel#1582</a></td><td>0</td><td>fix(grpo): mask the luspo loss elementwise to match TRL</td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/issues/1576">linkedin/Liger-Kernel#1576</a></td><td>0</td><td>fix(cutedsl): reuse SM90 scaled CE kernels across token counts</td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/issues/1580">linkedin/Liger-Kernel#1580</a></td><td>0</td><td>Add apply_liger_kernel_to_glm5_next (GLM-5.3-Flash): fused linear cross-entropy and RMSNorm</td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/issues/1577">linkedin/Liger-Kernel#1577</a></td><td>0</td><td>RoPE auto-dispatch prefers CuTe despite higher warmed host latency on SM90</td></tr><tr><td><a href="https://github.com/Blaizzy/mlx-vlm/issues/2623">Blaizzy/mlx-vlm#2623</a></td><td>0</td><td>Preserve inference state when resuming interrupted responses</td></tr><tr><td><a href="https://github.com/EvolvingLMMs-Lab/lmms-eval/issues/1623">EvolvingLMMs-Lab/lmms-eval#1623</a></td><td>0</td><td>fix(cache): include effective model identity in response keys</td></tr><tr><td><a href="https://github.com/vllm-project/llm-compressor/issues/3431">vllm-project/llm-compressor#3431</a></td><td>0</td><td>[Bug]: SpinQuantModifier crashes when transform_config is set and overwrites user mappings</td></tr><tr><td><a href="https://github.com/vllm-project/llm-compressor/issues/3429">vllm-project/llm-compressor#3429</a></td><td>0</td><td>[Bug]: Seven unit-offset RMSNorm classes are missing from the norm calibration registry, so smoothing changes those models&#x27; outputs</td></tr><tr><td><a href="https://github.com/PrimeIntellect-ai/prime-rl/issues/4144">PrimeIntellect-ai/prime-rl#4144</a></td><td>0</td><td>fix(qwen3_5): tie lm_head to input embeddings for tied checkpoints</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2337">unslothai/unsloth-zoo#2337</a></td><td>0</td><td>train_on_responses_only: jump to the next marker or boundary id with list.index instead of stepping every token</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/issues/2326">unslothai/unsloth-zoo#2326</a></td><td>0</td><td>NPU: avoid BF16 LoRA activation round trips while retaining FP32 weight paths</td></tr></table>
</details>
<h2>New monkey patches since last run (0)</h2>
<p>Something changed on their side or ours: check whether a transformers change forced it.</p>
<p>None.</p>
<h2>Most-patched transformers internals</h2>
<p>Patched by several repos = missing extension point (kernel hub / attention interface / hook).</p>
<table><tr><th>symbol</th><th>#repos</th><th>repos</th></tr><tr><td><code>*Attention.forward</code></td><td>8</td><td>FlagEmbedding, TensorRT-LLM, axolotl, LlamaFactory, ms-swift, unsloth, unsloth-zoo, verl</td></tr><tr><td><code>*ForCausalLM.forward</code></td><td>6</td><td>Automodel, axolotl, Liger-Kernel, ms-swift, unsloth, unsloth-zoo</td></tr><tr><td><code>*Model.forward</code></td><td>6</td><td>lmms-eval, axolotl, ms-swift, unsloth, unsloth-zoo, verl</td></tr><tr><td><code>*MLP.forward</code></td><td>5</td><td>TensorRT-LLM, axolotl, DeepSpeed, unsloth, verl</td></tr><tr><td><code>*DecoderLayer.forward</code></td><td>4</td><td>axolotl, LlamaFactory, unsloth, verl</td></tr><tr><td><code>*FlashAttention2</code></td><td>4</td><td>GPTQModel, ms-swift, sglang, unsloth-zoo</td></tr><tr><td><code>*Experts</code></td><td>3</td><td>axolotl, Liger-Kernel, unsloth-zoo</td></tr><tr><td><code>*Model._create_and_replace</code></td><td>3</td><td>axolotl, ms-swift, unsloth-zoo</td></tr><tr><td><code>*RMSNorm</code></td><td>3</td><td>axolotl, Liger-Kernel, unsloth</td></tr><tr><td><code>PreTrainedModel.from_pretrained</code></td><td>3</td><td>ColossalAI, ms-swift, unsloth</td></tr><tr><td><code>apply_rotary_pos_emb</code></td><td>3</td><td>axolotl, Liger-Kernel, verl</td></tr><tr><td><code>create_causal_mask</code></td><td>3</td><td>GPTQModel, ms-swift, unsloth-zoo</td></tr><tr><td><code>*Experts.forward</code></td><td>2</td><td>unsloth-zoo, verl</td></tr><tr><td><code>*FlashAttention2.forward</code></td><td>2</td><td>LlamaFactory, unsloth</td></tr><tr><td><code>*MLP</code></td><td>2</td><td>axolotl, Liger-Kernel</td></tr><tr><td><code>*Model._postprocess</code></td><td>2</td><td>ROLL, verl</td></tr><tr><td><code>*Model.load_adapter</code></td><td>2</td><td>ms-swift, unsloth-zoo</td></tr><tr><td><code>*Model.save_pretrained</code></td><td>2</td><td>unsloth, unsloth-zoo</td></tr><tr><td><code>*RMSNorm.forward</code></td><td>2</td><td>Automodel, verl</td></tr><tr><td><code>*SdpaAttention.forward</code></td><td>2</td><td>LlamaFactory, unsloth</td></tr><tr><td><code>*SparseMoeBlock</code></td><td>2</td><td>LlamaFactory, verl</td></tr><tr><td><code>CrossEntropyLoss</code></td><td>2</td><td>axolotl, Liger-Kernel</td></tr><tr><td><code>PreTrainedModel.enable_input_require_grads</code></td><td>2</td><td>ms-swift, unsloth</td></tr><tr><td><code>PreTrainedModel.gradient_checkpointing_enable</code></td><td>2</td><td>unsloth, unsloth-zoo</td></tr><tr><td><code>PreTrainedModel.post_init</code></td><td>2</td><td>sglang, unsloth</td></tr><tr><td><code>_init_weights</code></td><td>2</td><td>GPTQModel, ColossalAI</td></tr><tr><td><code>checkpoint</code></td><td>2</td><td>ms-swift, unsloth-zoo</td></tr><tr><td><code>dynamic_module_utils.check_imports</code></td><td>2</td><td>LlamaFactory, ktransformers</td></tr><tr><td><code>get_model_conversion_mapping</code></td><td>2</td><td>unsloth, unsloth-zoo</td></tr><tr><td><code>nn.CrossEntropyLoss</code></td><td>2</td><td>axolotl, Liger-Kernel</td></tr><tr><td><code>nn.LayerNorm</code></td><td>2</td><td>axolotl, Liger-Kernel</td></tr><tr><td><code>*Attention</code></td><td>1</td><td>GPTQModel</td></tr><tr><td><code>*Attention.ATTN_DONE</code></td><td>1</td><td>ColossalAI</td></tr><tr><td><code>*Attention.CU_SEQLENS</code></td><td>1</td><td>ColossalAI</td></tr><tr><td><code>*Attention.DEFAULT_BLOCK_SIZES</code></td><td>1</td><td>diffusers</td></tr><tr><td><code>*Attention.HALF_INDICES</code></td><td>1</td><td>ColossalAI</td></tr><tr><td><code>*Attention.INNER_RING_GROUP</code></td><td>1</td><td>ColossalAI</td></tr><tr><td><code>*Attention.INTER_RING_GROUP</code></td><td>1</td><td>ColossalAI</td></tr><tr><td><code>*Attention.SP_GROUP</code></td><td>1</td><td>ColossalAI</td></tr><tr><td><code>*Attention.SP_STREAM</code></td><td>1</td><td>ColossalAI</td></tr></table>
<h2>Per repo</h2>
<table><tr><th>repo</th><th>threads</th><th>patches</th><th>new patches</th></tr><tr><td><a href="https://github.com/ggml-org/llama.cpp">ggml-org/llama.cpp</a></td><td>5</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/vllm-project/vllm">vllm-project/vllm</a></td><td>100</td><td>6</td><td>0</td></tr><tr><td><a href="https://github.com/unslothai/unsloth">unslothai/unsloth</a></td><td>50</td><td>132</td><td>0</td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory">hiyouga/LlamaFactory</a></td><td>12</td><td>8</td><td>0</td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed">deepspeedai/DeepSpeed</a></td><td>17</td><td>11</td><td>0</td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI">hpcaitech/ColossalAI</a></td><td>0</td><td>14</td><td>0</td></tr><tr><td><a href="https://github.com/sgl-project/sglang">sgl-project/sglang</a></td><td>71</td><td>4</td><td>0</td></tr><tr><td><a href="https://github.com/huggingface/diffusers">huggingface/diffusers</a></td><td>13</td><td>3</td><td>0</td></tr><tr><td><a href="https://github.com/huggingface/lerobot">huggingface/lerobot</a></td><td>19</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/huggingface/open-r1">huggingface/open-r1</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/m-bain/whisperX">m-bain/whisperX</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/verl-project/verl">verl-project/verl</a></td><td>7</td><td>48</td><td>0</td></tr><tr><td><a href="https://github.com/huggingface/peft">huggingface/peft</a></td><td>18</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/huggingface/trl">huggingface/trl</a></td><td>11</td><td>1</td><td>0</td></tr><tr><td><a href="https://github.com/kvcache-ai/ktransformers">kvcache-ai/ktransformers</a></td><td>0</td><td>5</td><td>0</td></tr><tr><td><a href="https://github.com/huggingface/sentence-transformers">huggingface/sentence-transformers</a></td><td>6</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel">NVIDIA-NeMo/Automodel</a></td><td>31</td><td>10</td><td>0</td></tr><tr><td><a href="https://github.com/modelscope/ms-swift">modelscope/ms-swift</a></td><td>13</td><td>53</td><td>0</td></tr><tr><td><a href="https://github.com/dottxt-ai/outlines">dottxt-ai/outlines</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/NVIDIA/TensorRT-LLM">NVIDIA/TensorRT-LLM</a></td><td>6</td><td>2</td><td>0</td></tr><tr><td><a href="https://github.com/flairNLP/flair">flairNLP/flair</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/EleutherAI/lm-evaluation-harness">EleutherAI/lm-evaluation-harness</a></td><td>2</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl">axolotl-ai-cloud/axolotl</a></td><td>5</td><td>83</td><td>0</td></tr><tr><td><a href="https://github.com/FlagOpen/FlagEmbedding">FlagOpen/FlagEmbedding</a></td><td>4</td><td>2</td><td>0</td></tr><tr><td><a href="https://github.com/OpenRLHF/OpenRLHF">OpenRLHF/OpenRLHF</a></td><td>0</td><td>1</td><td>0</td></tr><tr><td><a href="https://github.com/huggingface/accelerate">huggingface/accelerate</a></td><td>5</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/THUDM/slime">THUDM/slime</a></td><td>1</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/bitsandbytes-foundation/bitsandbytes">bitsandbytes-foundation/bitsandbytes</a></td><td>1</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/ml-explore/mlx-lm">ml-explore/mlx-lm</a></td><td>6</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/arcee-ai/mergekit">arcee-ai/mergekit</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel">linkedin/Liger-Kernel</a></td><td>12</td><td>175</td><td>0</td></tr><tr><td><a href="https://github.com/Blaizzy/mlx-vlm">Blaizzy/mlx-vlm</a></td><td>4</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/EvolvingLMMs-Lab/lmms-eval">EvolvingLMMs-Lab/lmms-eval</a></td><td>2</td><td>2</td><td>0</td></tr><tr><td><a href="https://github.com/urchade/GLiNER">urchade/GLiNER</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/vllm-project/llm-compressor">vllm-project/llm-compressor</a></td><td>7</td><td>4</td><td>0</td></tr><tr><td><a href="https://github.com/allenai/open-instruct">allenai/open-instruct</a></td><td>2</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/huggingface/optimum">huggingface/optimum</a></td><td>2</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/embeddings-benchmark/mteb">embeddings-benchmark/mteb</a></td><td>4</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/alibaba/ROLL">alibaba/ROLL</a></td><td>0</td><td>7</td><td>0</td></tr><tr><td><a href="https://github.com/PrimeIntellect-ai/prime-rl">PrimeIntellect-ai/prime-rl</a></td><td>1</td><td>7</td><td>0</td></tr><tr><td><a href="https://github.com/intel/auto-round">intel/auto-round</a></td><td>4</td><td>2</td><td>0</td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel">ModelCloud/GPTQModel</a></td><td>1</td><td>15</td><td>0</td></tr><tr><td><a href="https://github.com/huggingface/optimum-quanto">huggingface/optimum-quanto</a></td><td>0</td><td>0</td><td>0</td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo">unslothai/unsloth-zoo</a></td><td>24</td><td>67</td><td>0</td></tr></table>
<details><summary>All 662 patches</summary>
<table><tr><th>where</th><th>patch</th></tr><tr><td><a href="https://github.com/EvolvingLMMs-Lab/lmms-eval/blob/HEAD/lmms_eval/models/simple/cambrians_vsc.py">EvolvingLMMs-Lab/lmms-eval/lmms_eval/models/simple/cambrians_vsc.py</a></td><td><code>CambrianQwenModel.forward = cambrian_qwen2_forward</code></td></tr><tr><td><a href="https://github.com/EvolvingLMMs-Lab/lmms-eval/blob/HEAD/lmms_eval/models/simple/cambrians_vsr.py">EvolvingLMMs-Lab/lmms-eval/lmms_eval/models/simple/cambrians_vsr.py</a></td><td><code>CambrianQwenModel.forward = cambrian_qwen2_forward</code></td></tr><tr><td><a href="https://github.com/FlagOpen/FlagEmbedding/blob/HEAD/research/llm_embedder/src/utils/llama_patch.py">FlagOpen/FlagEmbedding/research/llm_embedder/src/utils/llama_patch.py</a></td><td><code>transformers.models.llama.modeling_llama.LlamaAttention.forward = forward</code></td></tr><tr><td><a href="https://github.com/FlagOpen/FlagEmbedding/blob/HEAD/research/llm_embedder/src/utils/llama_patch.py">FlagOpen/FlagEmbedding/research/llm_embedder/src/utils/llama_patch.py</a></td><td><code>transformers.models.llama.modeling_llama.LlamaModel._prepare_decoder_attention_mask = (</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/models/base.py">ModelCloud/GPTQModel/gptqmodel/models/base.py</a></td><td><code>modeling_utils.check_support_param_buffer_assignment = check_support_param_buffer_assignment</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/models/definitions/intern_s2_preview.py">ModelCloud/GPTQModel/gptqmodel/models/definitions/intern_s2_preview.py</a></td><td><code>modeling_module.create_causal_mask = create_causal_mask_compat</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>PreTrainedModel.__getattr__ = __getattr__</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>PreTrainedModel._gptqmodel_legacy_tied_weights_patch = True</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>PreTrainedModel._gptqmodel_missing_all_tied_weights_patch = True</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>PreTrainedModel.get_expanded_tied_weights_keys = get_expanded_tied_weights_keys</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>PreTrainedModel.get_head_mask = get_head_mask</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>PreTrainedModel.is_parallelizable = False</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>modeling_auto.AutoModelForCausalLM.register = classmethod(patched_register)</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>modeling_llama.LlamaAttention = DeepSeekVLV2CompatLlamaAttention</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>modeling_llama.LlamaFlashAttention2 = DeepSeekVLV2CompatLlamaAttention</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>modeling_llama.LlamaFlashAttention2 = original_llama_attention</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>modeling_utils._init_weights = False</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>modeling_utils._init_weights = original_init_flag</code></td></tr><tr><td><a href="https://github.com/ModelCloud/GPTQModel/blob/HEAD/gptqmodel/utils/hf.py">ModelCloud/GPTQModel/gptqmodel/utils/hf.py</a></td><td><code>modeling_utils.is_flash_attn_2_available = is_flash_attn_2_available</code></td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/blob/HEAD/nemo_automodel/_transformers/auto_model.py">NVIDIA-NeMo/Automodel/nemo_automodel/_transformers/auto_model.py</a></td><td><code>PreTrainedModel._finalize_model_loading = staticmethod(_compat_finalize)</code></td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/blob/HEAD/nemo_automodel/_transformers/model_init.py">NVIDIA-NeMo/Automodel/nemo_automodel/_transformers/model_init.py</a></td><td><code>PreTrainedModel.get_init_context = classmethod(_patched_get_init_context)</code></td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/blob/HEAD/nemo_automodel/_transformers/utils.py">NVIDIA-NeMo/Automodel/nemo_automodel/_transformers/utils.py</a></td><td><code>Qwen3ForCausalLM.forward = _patched_forward # type: ignore[method-assign]</code></td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/blob/HEAD/nemo_automodel/components/models/beam/model.py">NVIDIA-NeMo/Automodel/nemo_automodel/components/models/beam/model.py</a></td><td><code>BeamForCausalLM.parallelizer = PARALLELIZER</code></td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/blob/HEAD/nemo_automodel/components/models/common/utils.py">NVIDIA-NeMo/Automodel/nemo_automodel/components/models/common/utils.py</a></td><td><code>TERMSNorm.forward = _patched_rmsnorm_forward</code></td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/blob/HEAD/nemo_automodel/components/models/deepseek_v4/model.py">NVIDIA-NeMo/Automodel/nemo_automodel/components/models/deepseek_v4/model.py</a></td><td><code>DeepseekV4ForCausalLM.parallelizer = PARALLELIZER</code></td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/blob/HEAD/nemo_automodel/components/models/deepseek_v41/model.py">NVIDIA-NeMo/Automodel/nemo_automodel/components/models/deepseek_v41/model.py</a></td><td><code>DeepseekV41ForCausalLM.parallelizer = PARALLELIZER</code></td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/blob/HEAD/nemo_automodel/components/models/nemotron_v3/model.py">NVIDIA-NeMo/Automodel/nemo_automodel/components/models/nemotron_v3/model.py</a></td><td><code>NemotronHForCausalLM.parallelizer = PARALLELIZER</code></td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/blob/HEAD/nemo_automodel/components/models/qwen3_5/model.py">NVIDIA-NeMo/Automodel/nemo_automodel/components/models/qwen3_5/model.py</a></td><td><code>Qwen3_5ForCausalLM.parallelizer = PARALLELIZER</code></td></tr><tr><td><a href="https://github.com/NVIDIA-NeMo/Automodel/blob/HEAD/nemo_automodel/shared/transformers_patches.py">NVIDIA-NeMo/Automodel/nemo_automodel/shared/transformers_patches.py</a></td><td><code>modeling_t5.T5LayerNorm = _NativeT5LayerNorm</code></td></tr><tr><td><a href="https://github.com/NVIDIA/TensorRT-LLM/blob/HEAD/tensorrt_llm/tools/layer_wise_benchmarks/mark_utils.py">NVIDIA/TensorRT-LLM/tensorrt_llm/tools/layer_wise_benchmarks/mark_utils.py</a></td><td><code>GatedMLP.forward = nvtx.annotate(&quot;GatedMLP&quot;)(GatedMLP.forward)</code></td></tr><tr><td><a href="https://github.com/NVIDIA/TensorRT-LLM/blob/HEAD/tensorrt_llm/tools/layer_wise_benchmarks/mark_utils.py">NVIDIA/TensorRT-LLM/tensorrt_llm/tools/layer_wise_benchmarks/mark_utils.py</a></td><td><code>KimiKDALinearAttention.forward = nvtx.annotate(&quot;KimiKDALinearAttention&quot;)(</code></td></tr><tr><td><a href="https://github.com/OpenRLHF/OpenRLHF/blob/HEAD/openrlhf/utils/deepspeed/deepspeed.py">OpenRLHF/OpenRLHF/openrlhf/utils/deepspeed/deepspeed.py</a></td><td><code>transformers.modeling_flash_attention_utils.deterministic_g = True</code></td></tr><tr><td><a href="https://github.com/PrimeIntellect-ai/prime-rl/blob/HEAD/src/prime_rl/inference/patches.py">PrimeIntellect-ai/prime-rl/src/prime_rl/inference/patches.py</a></td><td><code>Glm5NextForCausalLM.process_weights_after_loading = process_weights_after_loading</code></td></tr><tr><td><a href="https://github.com/PrimeIntellect-ai/prime-rl/blob/HEAD/src/prime_rl/inference/patches.py">PrimeIntellect-ai/prime-rl/src/prime_rl/inference/patches.py</a></td><td><code>LoRAModel.from_local_checkpoint = classmethod(_patched_from_local_checkpoint)</code></td></tr><tr><td><a href="https://github.com/PrimeIntellect-ai/prime-rl/blob/HEAD/src/prime_rl/inference/patches.py">PrimeIntellect-ai/prime-rl/src/prime_rl/inference/patches.py</a></td><td><code>MiniMaxM2ForCausalLM.hf_to_vllm_mapper = WeightsMapper(</code></td></tr><tr><td><a href="https://github.com/PrimeIntellect-ai/prime-rl/blob/HEAD/src/prime_rl/inference/vllm/qwen38_weight_loading.py">PrimeIntellect-ai/prime-rl/src/prime_rl/inference/vllm/qwen38_weight_loading.py</a></td><td><code>Qwen4ExpNGramEmbedding.load_weights = load_weights</code></td></tr><tr><td><a href="https://github.com/PrimeIntellect-ai/prime-rl/blob/HEAD/src/prime_rl/trainer/models/gpt_oss/attention.py">PrimeIntellect-ai/prime-rl/src/prime_rl/trainer/models/gpt_oss/attention.py</a></td><td><code>GptOssAttention.compute_attention = ulysses_attention</code></td></tr><tr><td><a href="https://github.com/PrimeIntellect-ai/prime-rl/blob/HEAD/src/prime_rl/trainer/models/layers/ulysses_attn.py">PrimeIntellect-ai/prime-rl/src/prime_rl/trainer/models/layers/ulysses_attn.py</a></td><td><code>AfmoeFlashAttention._compute_attention = _ulysses_compute_attention</code></td></tr><tr><td><a href="https://github.com/PrimeIntellect-ai/prime-rl/blob/HEAD/src/prime_rl/trainer/models/layers/ulysses_attn.py">PrimeIntellect-ai/prime-rl/src/prime_rl/trainer/models/layers/ulysses_attn.py</a></td><td><code>FlashAttention._compute_attention = _ulysses_compute_attention</code></td></tr><tr><td><a href="https://github.com/alibaba/ROLL/blob/HEAD/roll/models/model_providers.py">alibaba/ROLL/roll/models/model_providers.py</a></td><td><code>AutoModelForCausalLMWithValueHead.post_init = no_set_device_hook_post_init</code></td></tr><tr><td><a href="https://github.com/alibaba/ROLL/blob/HEAD/roll/third_party/megatron/mtp_patcher.py">alibaba/ROLL/roll/third_party/megatron/mtp_patcher.py</a></td><td><code>GPTModel._original_postprocess = original_gpt_postprocess</code></td></tr><tr><td><a href="https://github.com/alibaba/ROLL/blob/HEAD/roll/third_party/megatron/mtp_patcher.py">alibaba/ROLL/roll/third_party/megatron/mtp_patcher.py</a></td><td><code>GPTModel._postprocess = patched_gpt_postprocess</code></td></tr><tr><td><a href="https://github.com/alibaba/ROLL/blob/HEAD/roll/third_party/vllm/gdn_patcher.py">alibaba/ROLL/roll/third_party/vllm/gdn_patcher.py</a></td><td><code>GDNAttentionMetadataBuilder._original_build = original_build</code></td></tr><tr><td><a href="https://github.com/alibaba/ROLL/blob/HEAD/roll/third_party/vllm/gdn_patcher.py">alibaba/ROLL/roll/third_party/vllm/gdn_patcher.py</a></td><td><code>GDNAttentionMetadataBuilder.build = patched_build</code></td></tr><tr><td><a href="https://github.com/alibaba/ROLL/blob/HEAD/roll/utils/context_parallel/monkey_patch.py">alibaba/ROLL/roll/utils/context_parallel/monkey_patch.py</a></td><td><code>Qwen2Model._update_causal_mask = _update_causal_mask</code></td></tr><tr><td><a href="https://github.com/alibaba/ROLL/blob/HEAD/roll/utils/context_parallel/monkey_patch.py">alibaba/ROLL/roll/utils/context_parallel/monkey_patch.py</a></td><td><code>Qwen2Model._update_causal_mask = old_update_causal_mask</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/expert_parallel/experts_fn.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/expert_parallel/experts_fn.py</a></td><td><code>PreTrainedModel.get_correct_experts_implementation = patched # type: ignore[assignment]</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/kernels/libs/scattermoe_lora/experts.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/kernels/libs/scattermoe_lora/experts.py</a></td><td><code>PreTrainedModel.get_correct_experts_implementation = _patched_get_correct</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/kernels/libs/scattermoe_lora/nvfp4_moe_loading.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/kernels/libs/scattermoe_lora/nvfp4_moe_loading.py</a></td><td><code>PreTrainedModel._initialize_missing_keys = patched</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/glm4_moe_lite.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/glm4_moe_lite.py</a></td><td><code>modeling_glm4_moe_lite.Glm4MoeLiteMLP.forward = LigerSwiGLUMLP.forward</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/glm4_moe_lite.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/glm4_moe_lite.py</a></td><td><code>modeling_glm4_moe_lite.Glm4MoeLiteRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/glm4_moe_lite.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/glm4_moe_lite.py</a></td><td><code>modeling_glm4_moe_lite.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/glm4_moe_lite.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/glm4_moe_lite.py</a></td><td><code>modeling_glm4_moe_lite.apply_rotary_pos_emb_interleave = (</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/qwen3_5.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/qwen3_5.py</a></td><td><code>modeling_qwen3_5.Qwen3_5ForCausalLM.forward = lce_forward</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/qwen3_5.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/qwen3_5.py</a></td><td><code>modeling_qwen3_5.Qwen3_5MLP = _liger_swiglu_mlp_wrapper</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/qwen3_5.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/qwen3_5.py</a></td><td><code>modeling_qwen3_5.Qwen3_5RMSNorm = LigerRMSNormForQwen3_5</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/qwen3_5.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/qwen3_5.py</a></td><td><code>modeling_qwen3_5.Qwen3_5RMSNormGated = FusedRMSNormGated</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/qwen3_5.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/qwen3_5.py</a></td><td><code>modeling_qwen3_5.nn.LayerNorm = LigerLayerNorm</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/qwen3_5_moe.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/qwen3_5_moe.py</a></td><td><code>modeling_mod.Qwen3_5MoeForCausalLM.forward = lce_forward</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/qwen3_5_moe.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/qwen3_5_moe.py</a></td><td><code>modeling_mod.Qwen3_5MoeMLP = _liger_swiglu_mlp_wrapper</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/qwen3_5_moe.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/qwen3_5_moe.py</a></td><td><code>modeling_mod.Qwen3_5MoeRMSNorm = LigerRMSNormForQwen3_5Moe</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/qwen3_5_moe.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/qwen3_5_moe.py</a></td><td><code>modeling_mod.Qwen3_5MoeRMSNormGated = FusedRMSNormGated</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/models/qwen3_5_moe.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/models/qwen3_5_moe.py</a></td><td><code>modeling_mod.nn.LayerNorm = LigerLayerNorm</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_cohere_compass.CohereCompassMLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_cohere_compass.nn.LayerNorm = LigerLayerNorm</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_gemma4_unified.Gemma4UnifiedRMSNorm = (</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_gemma4_unified.Gemma4UnifiedTextMLP = _LigerGemma4UnifiedMLP</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_gemma4_unified.nn.CrossEntropyLoss = LigerCrossEntropyLoss</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_gemma4_unified.nn.LayerNorm = LigerLayerNorm</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_jamba.JambaForCausalLM.forward = jamba_lce_forward</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_jamba.JambaMLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_jamba.JambaRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_jamba.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_jamba.nn.LayerNorm = LigerLayerNorm</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_ministral3.Ministral3MLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_ministral3.Ministral3RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/integrations/liger/plugin.py">axolotl-ai-cloud/axolotl/src/axolotl/integrations/liger/plugin.py</a></td><td><code>modeling_ministral3.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/loaders/adapter.py">axolotl-ai-cloud/axolotl/src/axolotl/loaders/adapter.py</a></td><td><code>LoraModel._axolotl_clippable_patched = True</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/loaders/adapter.py">axolotl-ai-cloud/axolotl/src/axolotl/loaders/adapter.py</a></td><td><code>LoraModel._create_and_replace = _patched</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/loaders/model.py">axolotl-ai-cloud/axolotl/src/axolotl/loaders/model.py</a></td><td><code>transformers.integrations.deepspeed.is_deepspeed_zero3_enabled = lambda: (</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/loaders/model.py">axolotl-ai-cloud/axolotl/src/axolotl/loaders/model.py</a></td><td><code>transformers.modeling_utils.is_deepspeed_zero3_enabled = lambda: True</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/accelerate/fsdp2.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/accelerate/fsdp2.py</a></td><td><code>PreTrainedModel._adjust_tied_keys_with_tied_pointers = (</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/accelerate/fsdp2.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/accelerate/fsdp2.py</a></td><td><code>PreTrainedModel._initialize_missing_keys = _patched_initialize_missing_keys</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/accelerate/fsdp2.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/accelerate/fsdp2.py</a></td><td><code>PreTrainedModel._move_missing_keys_from_meta_to_device = _patched_move_missing_keys</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/attention/flex_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/attention/flex_attn.py</a></td><td><code>transformers.integrations.flex_attention.WrappedFlexAttention = WrappedFlexAttention</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/btlm_attn_hijack_flash.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/btlm_attn_hijack_flash.py</a></td><td><code>modeling_btlm.BTLMAttention._attn = flashattn_attn</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/kernelize_fixes.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/kernelize_fixes.py</a></td><td><code>PreTrainedModel.set_use_kernels = _ORIG_SET_USE_KERNELS</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/kernelize_fixes.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/kernelize_fixes.py</a></td><td><code>PreTrainedModel.set_use_kernels = set_use_kernels</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/llama_attn_hijack_flash.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/llama_attn_hijack_flash.py</a></td><td><code>transformers.loss.loss_utils.fixed_cross_entropy = fa2_fixed_cross_entropy</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/llama_attn_hijack_xformers.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/llama_attn_hijack_xformers.py</a></td><td><code>transformers.models.llama.modeling_llama.LlamaAttention.forward = xformers_forward</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/loss/chunked.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/loss/chunked.py</a></td><td><code>transformers.loss.loss_utils.ForCausalLMLoss = for_causal_lm_chunked_loss</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/mistral_attn_hijack_flash.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/mistral_attn_hijack_flash.py</a></td><td><code>transformers.models.mistral.modeling_mistral.CrossEntropyLoss = partial(</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/mixtral/__init__.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/mixtral/__init__.py</a></td><td><code>MixtralBlockSparseTop2MLP.forward = mlp_forward</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/falcon_h1/modeling.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/falcon_h1/modeling.py</a></td><td><code>FalconH1DecoderLayer.forward = patched_decoder_forward</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/gemma4/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/gemma4/fused_attn.py</a></td><td><code>Gemma4TextAttention.forward = _make_fused_forward(original_forward)</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/gemma4/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/gemma4/fused_attn.py</a></td><td><code>Gemma4TextDecoderLayer.__call__ = patched_call</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/gemma4/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/gemma4/fused_attn.py</a></td><td><code>Gemma4TextDecoderLayer._axolotl_shared_kv_patched = True</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/gemma4_unified/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/gemma4_unified/fused_attn.py</a></td><td><code>Gemma4UnifiedTextAttention._axolotl_fused_attn_patched = True</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/gemma4_unified/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/gemma4_unified/fused_attn.py</a></td><td><code>Gemma4UnifiedTextAttention.forward = _make_fused_forward(original_forward)</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/gemma4_unified/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/gemma4_unified/fused_attn.py</a></td><td><code>Gemma4UnifiedTextDecoderLayer.__call__ = patched_call</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/gemma4_unified/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/gemma4_unified/fused_attn.py</a></td><td><code>Gemma4UnifiedTextDecoderLayer._axolotl_shared_kv_patched = True</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/glm4_moe_lite/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/glm4_moe_lite/fused_attn.py</a></td><td><code>Glm4MoeLiteAttention._axolotl_fused_attn_patched = True</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/glm4_moe_lite/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/glm4_moe_lite/fused_attn.py</a></td><td><code>Glm4MoeLiteAttention.forward = _make_fused_forward(Glm4MoeLiteAttention.forward)</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/llama4/modeling.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/llama4/modeling.py</a></td><td><code>modeling_llama4.Llama4TextExperts = Llama4TextExperts</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/llama4/modeling.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/llama4/modeling.py</a></td><td><code>modeling_llama4.Llama4TextExperts = old_lamma_4_text_experts</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/pixtral/modeling_flash_attention_utils.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/pixtral/modeling_flash_attention_utils.py</a></td><td><code>modeling_flash_attention_utils._is_packed_sequence = fixed_is_packed_sequence</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/pixtral/modeling_flash_attention_utils.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/pixtral/modeling_flash_attention_utils.py</a></td><td><code>modeling_flash_attention_utils._is_packed_sequence = old_fn</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3/fused_attn.py</a></td><td><code>Qwen3Attention._axolotl_fused_attn_patched = True</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3/fused_attn.py</a></td><td><code>Qwen3Attention.forward = _make_fused_forward()</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3_5/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3_5/fused_attn.py</a></td><td><code>Qwen3_5Attention._axolotl_fused_attn_patched = True</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3_5/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3_5/fused_attn.py</a></td><td><code>Qwen3_5Attention.forward = _make_fused_forward()</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3_5_moe/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3_5_moe/fused_attn.py</a></td><td><code>Qwen3_5MoeAttention._axolotl_fused_attn_patched = True</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3_5_moe/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3_5_moe/fused_attn.py</a></td><td><code>Qwen3_5MoeAttention.forward = _make_fused_forward()</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3_moe/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3_moe/fused_attn.py</a></td><td><code>Qwen3MoeAttention._axolotl_fused_attn_patched = True</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3_moe/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3_moe/fused_attn.py</a></td><td><code>Qwen3MoeAttention.forward = _make_fused_forward()</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3_next/modeling.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3_next/modeling.py</a></td><td><code>Qwen3NextDecoderLayer.forward = original_decoder_forward</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3_next/modeling.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3_next/modeling.py</a></td><td><code>Qwen3NextDecoderLayer.forward = patched_decoder_forward</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3_vl/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3_vl/fused_attn.py</a></td><td><code>Qwen3VLTextAttention._axolotl_fused_attn_patched = True</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/models/qwen3_vl/fused_attn.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/models/qwen3_vl/fused_attn.py</a></td><td><code>Qwen3VLTextAttention.forward = _make_fused_forward()</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/moe_quant.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/moe_quant.py</a></td><td><code>transformers.core_model_loading.set_param_for_module = _patched_set_param_for_module</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/moe_quant.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/moe_quant.py</a></td><td><code>transformers.modeling_utils.caching_allocator_warmup = _noop_warmup</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/multipack.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/multipack.py</a></td><td><code>modeling_arch._get_unpad_data = get_unpad_data</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/peft/nf4.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/peft/nf4.py</a></td><td><code>LoraModel.merge_and_unload = merge_and_unload</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/quantized_init.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/quantized_init.py</a></td><td><code>PreTrainedModel._initialize_weights = _initialize_weights</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/stablelm_attn_hijack_flash.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/stablelm_attn_hijack_flash.py</a></td><td><code>modeling_stablelm.Attention.forward = flashattn_attn</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/stablelm_attn_hijack_flash.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/stablelm_attn_hijack_flash.py</a></td><td><code>modeling_stablelm.DecoderLayer.forward = decoder_layer_forward</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/stablelm_attn_hijack_flash.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/stablelm_attn_hijack_flash.py</a></td><td><code>modeling_stablelm.StableLMEpochModel.forward = stablelm_model_forward</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/transformers_fa_utils.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/transformers_fa_utils.py</a></td><td><code>transformers.modeling_flash_attention_utils.fa_peft_integration_check = partial(</code></td></tr><tr><td><a href="https://github.com/axolotl-ai-cloud/axolotl/blob/HEAD/src/axolotl/monkeypatch/transformers_num_parameters.py">axolotl-ai-cloud/axolotl/src/axolotl/monkeypatch/transformers_num_parameters.py</a></td><td><code>PreTrainedModel.resize_token_embeddings = resize_token_embeddings # type: ignore[method-assign]</code></td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/blob/HEAD/deepspeed/inference/engine.py">deepspeedai/DeepSpeed/deepspeed/inference/engine.py</a></td><td><code>DeepSpeedSelfAttention.num_layers = 0</code></td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/blob/HEAD/deepspeed/linear/context_manager.py">deepspeedai/DeepSpeed/deepspeed/linear/context_manager.py</a></td><td><code>transformers.AutoModelForCausalLM.from_config = from_config</code></td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/blob/HEAD/deepspeed/linear/context_manager.py">deepspeedai/DeepSpeed/deepspeed/linear/context_manager.py</a></td><td><code>transformers.AutoModelForCausalLM.from_config = self._orig_causallm_config</code></td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/blob/HEAD/deepspeed/linear/context_manager.py">deepspeedai/DeepSpeed/deepspeed/linear/context_manager.py</a></td><td><code>transformers.AutoModelForCausalLM.from_pretrained = from_pretrained</code></td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/blob/HEAD/deepspeed/linear/context_manager.py">deepspeedai/DeepSpeed/deepspeed/linear/context_manager.py</a></td><td><code>transformers.AutoModelForCausalLM.from_pretrained = self._orig_causallm_pretrained</code></td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/blob/HEAD/deepspeed/ops/transformer/inference/ds_attention.py">deepspeedai/DeepSpeed/deepspeed/ops/transformer/inference/ds_attention.py</a></td><td><code>DeepSpeedSelfAttention._qkv_buffers = [</code></td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/blob/HEAD/deepspeed/ops/transformer/inference/ds_attention.py">deepspeedai/DeepSpeed/deepspeed/ops/transformer/inference/ds_attention.py</a></td><td><code>DeepSpeedSelfAttention.num_layers = DeepSpeedSelfAttention.num_layers + 1</code></td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/blob/HEAD/deepspeed/ops/transformer/inference/ds_mlp.py">deepspeedai/DeepSpeed/deepspeed/ops/transformer/inference/ds_mlp.py</a></td><td><code>DeepSpeedMLP._inter_w_buffers = [</code></td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/blob/HEAD/deepspeed/ops/transformer/inference/triton/attention.py">deepspeedai/DeepSpeed/deepspeed/ops/transformer/inference/triton/attention.py</a></td><td><code>TritonSelfAttention.num_layers = TritonSelfAttention.num_layers + 1</code></td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/blob/HEAD/deepspeed/runtime/sequence_parallel/ulysses_sp.py">deepspeedai/DeepSpeed/deepspeed/runtime/sequence_parallel/ulysses_sp.py</a></td><td><code>modeling_llama.LlamaMLP.forward = mlp_forward_new</code></td></tr><tr><td><a href="https://github.com/deepspeedai/DeepSpeed/blob/HEAD/deepspeed/runtime/sequence_parallel/ulysses_sp.py">deepspeedai/DeepSpeed/deepspeed/runtime/sequence_parallel/ulysses_sp.py</a></td><td><code>modeling_llama.LlamaMLP.forward = tiled_mlp_forward</code></td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/blob/HEAD/src/llamafactory/extras/misc.py">hiyouga/LlamaFactory/src/llamafactory/extras/misc.py</a></td><td><code>transformers.dynamic_module_utils.check_imports = get_relative_imports</code></td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/blob/HEAD/src/llamafactory/model/model_utils/longlora.py">hiyouga/LlamaFactory/src/llamafactory/model/model_utils/longlora.py</a></td><td><code>LlamaAttention.forward = llama_attention_forward</code></td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/blob/HEAD/src/llamafactory/model/model_utils/longlora.py">hiyouga/LlamaFactory/src/llamafactory/model/model_utils/longlora.py</a></td><td><code>LlamaFlashAttention2.forward = llama_flash_attention_2_forward</code></td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/blob/HEAD/src/llamafactory/model/model_utils/longlora.py">hiyouga/LlamaFactory/src/llamafactory/model/model_utils/longlora.py</a></td><td><code>LlamaSdpaAttention.forward = llama_sdpa_attention_forward</code></td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/blob/HEAD/src/llamafactory/model/model_utils/visual.py">hiyouga/LlamaFactory/src/llamafactory/model/model_utils/visual.py</a></td><td><code>transformers.models.llava.modeling_llava.LlavaMultiModalProjector = LlavaMultiModalProjectorForYiVL</code></td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/blob/HEAD/src/llamafactory/model/patcher.py">hiyouga/LlamaFactory/src/llamafactory/model/patcher.py</a></td><td><code>Qwen3_5DecoderLayer.forward = _patched_decoder_forward</code></td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/blob/HEAD/src/llamafactory/model/patcher.py">hiyouga/LlamaFactory/src/llamafactory/model/patcher.py</a></td><td><code>Qwen3_5MoeDecoderLayer.forward = _patched_decoder_forward</code></td></tr><tr><td><a href="https://github.com/hiyouga/LlamaFactory/blob/HEAD/src/llamafactory/model/patcher.py">hiyouga/LlamaFactory/src/llamafactory/model/patcher.py</a></td><td><code>modeling_qwen3_omni_moe.Qwen3OmniMoeThinkerTextSparseMoeBlock = Qwen3OmniMoeThinkerTextSparseMoeBlock</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/lazy/pretrained.py">hpcaitech/ColossalAI/colossalai/lazy/pretrained.py</a></td><td><code>PreTrainedModel.from_pretrained = classmethod(PretrainedManager.old_from_pretrained)</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/lazy/pretrained.py">hpcaitech/ColossalAI/colossalai/lazy/pretrained.py</a></td><td><code>PreTrainedModel.from_pretrained = new_from_pretrained</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/legacy/inference/quant/smoothquant/models/base_model.py">hpcaitech/ColossalAI/colossalai/legacy/inference/quant/smoothquant/models/base_model.py</a></td><td><code>transformers.modeling_utils._init_weights = False</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/shardformer/layer/attn.py">hpcaitech/ColossalAI/colossalai/shardformer/layer/attn.py</a></td><td><code>ColoAttention._flash_kernel_dispatch = ColoAttention._flash_kernel_dispatch.load()</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/shardformer/layer/attn.py">hpcaitech/ColossalAI/colossalai/shardformer/layer/attn.py</a></td><td><code>ColoAttention._flash_kernel_dispatch = FlashAttentionDaoLoader()</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/shardformer/layer/attn.py">hpcaitech/ColossalAI/colossalai/shardformer/layer/attn.py</a></td><td><code>ColoAttention._kernel_dispatch_map = {</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/shardformer/layer/attn.py">hpcaitech/ColossalAI/colossalai/shardformer/layer/attn.py</a></td><td><code>RingAttention.ATTN_DONE = torch.cuda.Event()</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/shardformer/layer/attn.py">hpcaitech/ColossalAI/colossalai/shardformer/layer/attn.py</a></td><td><code>RingAttention.CU_SEQLENS = cu_seqlens</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/shardformer/layer/attn.py">hpcaitech/ColossalAI/colossalai/shardformer/layer/attn.py</a></td><td><code>RingAttention.HALF_INDICES = (half_idx_front, half_idx_back)</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/shardformer/layer/attn.py">hpcaitech/ColossalAI/colossalai/shardformer/layer/attn.py</a></td><td><code>RingAttention.INNER_RING_GROUP = inner_ring_group</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/shardformer/layer/attn.py">hpcaitech/ColossalAI/colossalai/shardformer/layer/attn.py</a></td><td><code>RingAttention.INTER_RING_GROUP = inter_ring_group</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/shardformer/layer/attn.py">hpcaitech/ColossalAI/colossalai/shardformer/layer/attn.py</a></td><td><code>RingAttention.SP_GROUP = sp_group</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/shardformer/layer/attn.py">hpcaitech/ColossalAI/colossalai/shardformer/layer/attn.py</a></td><td><code>RingAttention.SP_STREAM = torch.cuda.Stream()</code></td></tr><tr><td><a href="https://github.com/hpcaitech/ColossalAI/blob/HEAD/colossalai/shardformer/layer/attn.py">hpcaitech/ColossalAI/colossalai/shardformer/layer/attn.py</a></td><td><code>RingAttention.TOTAL_SEQLEN = b * sq</code></td></tr><tr><td><a href="https://github.com/huggingface/diffusers/blob/HEAD/examples/research_projects/pytorch_xla/inference/flux/flux_inference.py">huggingface/diffusers/examples/research_projects/pytorch_xla/inference/flux/flux_inference.py</a></td><td><code>FlashAttention.DEFAULT_BLOCK_SIZES = {</code></td></tr><tr><td><a href="https://github.com/huggingface/diffusers/blob/HEAD/examples/research_projects/pytorch_xla/inference/flux/flux_inference_spmd.py">huggingface/diffusers/examples/research_projects/pytorch_xla/inference/flux/flux_inference_spmd.py</a></td><td><code>FlashAttention.DEFAULT_BLOCK_SIZES = {</code></td></tr><tr><td><a href="https://github.com/huggingface/diffusers/blob/HEAD/src/diffusers/pipelines/prx/__init__.py">huggingface/diffusers/src/diffusers/pipelines/prx/__init__.py</a></td><td><code>transformers.T5GemmaEncoder = T5GemmaEncoder</code></td></tr><tr><td><a href="https://github.com/huggingface/trl/blob/HEAD/trl/_compat.py">huggingface/trl/trl/_compat.py</a></td><td><code>transformers.cache_utils.HybridCache = Cache</code></td></tr><tr><td><a href="https://github.com/intel/auto-round/blob/HEAD/auto_round/modeling/unfused_moe/__init__.py">intel/auto-round/auto_round/modeling/unfused_moe/__init__.py</a></td><td><code>transformers.modeling_utils.get_checkpoint_conversion_mapping = get_checkpoint_conversion_mapping_ar</code></td></tr><tr><td><a href="https://github.com/intel/auto-round/blob/HEAD/auto_round/utils/common.py">intel/auto-round/auto_round/utils/common.py</a></td><td><code>transformers.modeling_utils.no_init_weights = no_init_weights</code></td></tr><tr><td><a href="https://github.com/kvcache-ai/ktransformers/blob/HEAD/archive/kt-sft/ktransformers/operators/dynamic_attention.py">kvcache-ai/ktransformers/archive/kt-sft/ktransformers/operators/dynamic_attention.py</a></td><td><code>DynamicScaledDotProductAttention.cpu_infer = CPUInfer(threads_num)</code></td></tr><tr><td><a href="https://github.com/kvcache-ai/ktransformers/blob/HEAD/archive/kt-sft/ktransformers/operators/models.py">kvcache-ai/ktransformers/archive/kt-sft/ktransformers/operators/models.py</a></td><td><code>KLlamaModel.dynamic_sdpa = DynamicScaledDotProductAttention(</code></td></tr><tr><td><a href="https://github.com/kvcache-ai/ktransformers/blob/HEAD/archive/kt-sft/ktransformers/sft/metrics_utils/misc.py">kvcache-ai/ktransformers/archive/kt-sft/ktransformers/sft/metrics_utils/misc.py</a></td><td><code>transformers.dynamic_module_utils.check_imports = get_relative_imports</code></td></tr><tr><td><a href="https://github.com/kvcache-ai/ktransformers/blob/HEAD/archive/ktransformers/operators/dynamic_attention.py">kvcache-ai/ktransformers/archive/ktransformers/operators/dynamic_attention.py</a></td><td><code>DynamicScaledDotProductAttention.cpu_infer = CPUInfer(threads_num)</code></td></tr><tr><td><a href="https://github.com/kvcache-ai/ktransformers/blob/HEAD/archive/ktransformers/operators/models.py">kvcache-ai/ktransformers/archive/ktransformers/operators/models.py</a></td><td><code>KLlamaModel.dynamic_sdpa = DynamicScaledDotProductAttention(</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/llama4_rope.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/llama4_rope.py</a></td><td><code>modeling_module.apply_rotary_emb = liger_llama4_text_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/llama4_rope.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/llama4_rope.py</a></td><td><code>modeling_module.vision_apply_rotary_emb = liger_llama4_vision_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>Gemma4TextForCausalLM.forward = causal_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_deepseek_v2.DeepseekV2Experts = LigerExperts</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_deepseek_v2.DeepseekV2ForCausalLM.forward = deepseek_v2_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_deepseek_v2.DeepseekV2MLP = LigerQwen3MoeSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_deepseek_v2.DeepseekV2RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_deepseek_v3.DeepseekV3Experts = LigerExperts</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_deepseek_v3.DeepseekV3ForCausalLM.forward = deepseek_v3_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_deepseek_v3.DeepseekV3MLP = LigerQwen3MoeSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_deepseek_v3.DeepseekV3RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_deepseek_v4.DeepseekV4ForCausalLM.forward = deepseek_v4_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_deepseek_v4.DeepseekV4RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_exaone4.Exaone4ForCausalLM.forward = exaone4_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_exaone4.Exaone4MLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_exaone4.Exaone4RMSNorm = Exaone4LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_exaone4.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_falcon_h1.FalconH1ForCausalLM.forward = falcon_h1_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_falcon_h1.FalconH1MLP = LigerFalconH1SwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_falcon_h1.FalconH1RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_falcon_h1.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma.GemmaForCausalLM.forward = gemma_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma.GemmaMLP = LigerGEGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma.GemmaRMSNorm = LigerRMSNormForGemma</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma2.Gemma2ForCausalLM.forward = gemma2_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma2.Gemma2MLP = LigerGEGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma2.Gemma2RMSNorm = LigerRMSNormForGemma2</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma2.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma3.Gemma3ForCausalLM.forward = causal_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma3.Gemma3ForConditionalGeneration.forward = multimodal_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma3.Gemma3MLP = LigerGEGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma3.Gemma3RMSNorm = LigerRMSNormForGemma3</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma3.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma3.nn.CrossEntropyLoss = LigerCrossEntropyLoss</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma4.Gemma4ForCausalLM.forward = causal_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma4.Gemma4ForConditionalGeneration.forward = multimodal_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma4.Gemma4RMSNorm = LigerRMSNormForGemma4</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gemma4.Gemma4TextMLP = LigerGEGLUMLPForGemma4</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_glm4.Glm4ForCausalLM.forward = glm4_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_glm4.Glm4MLP = LigerPhi3SwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_glm4.Glm4RMSNorm = LigerRMSNormForGlm4</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_glm4v.Glm4vForConditionalGeneration.forward = glm4v_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_glm4v.Glm4vRMSNorm = LigerRMSNormForGlm4</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_glm4v_moe.Glm4vMoeForConditionalGeneration.forward = glm4v_moe_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_glm4v_moe.Glm4vMoeRMSNorm = LigerRMSNormForGlm4</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_glm4v_moe.Glm4vMoeTextNaiveMoe = LigerExperts</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_glm4v_moe.Glm4vMoeTextRMSNorm = LigerRMSNormForGlm4</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gpt_oss.GptOssForCausalLM.forward = gpt_oss_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gpt_oss.GptOssRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_gpt_oss.apply_rotary_pos_emb = liger_rotary_pos_emb_gpt_oss</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_granite.GraniteMLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_granite.GraniteRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_granite.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_hunyuan_v1_dense.HunYuanDenseV1ForCausalLM.forward = hunyuan_v1_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_hunyuan_v1_dense.HunYuanDenseV1MLP = LigerHunyuanV1SwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_hunyuan_v1_dense.HunYuanDenseV1RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_hunyuan_v1_dense.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_hunyuan_v1_moe.HunYuanMoEV1Experts = LigerExperts</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_hunyuan_v1_moe.HunYuanMoEV1ForCausalLM.forward = hunyuan_v1_moe_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_hunyuan_v1_moe.HunYuanMoEV1RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_hunyuan_v1_moe.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_internvl.InternVLForConditionalGeneration.forward = internvl_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_internvl.InternVLVisionRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_internvl.nn.LayerNorm = LigerLayerNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_llama.LlamaForCausalLM.forward = llama_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_llama.LlamaMLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_llama.LlamaRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_llama.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_llama4.CrossEntropyLoss = LigerCrossEntropyLoss</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_llama4.Llama4ForCausalLM.forward = llama4_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_llama4.Llama4TextMLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_llama4.Llama4TextRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_llava.LlavaForConditionalGeneration.forward = llava_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_ministral.CrossEntropyLoss = LigerCrossEntropyLoss</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_ministral.MinistralForCausalLM.forward = ministral_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_ministral.MinistralMLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_ministral.MinistralRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_ministral.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mistral.CrossEntropyLoss = LigerCrossEntropyLoss</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mistral.MistralForCausalLM.forward = mistral_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mistral.MistralMLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mistral.MistralRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mistral.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mixtral.MixtralExperts = LigerExperts</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mixtral.MixtralForCausalLM.forward = mixtral_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mixtral.MixtralRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mixtral.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mllama.MllamaForCausalLM.forward = mllama_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mllama.MllamaTextMLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mllama.MllamaTextRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mllama.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_mllama.nn.LayerNorm = LigerLayerNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_muse_glimmer.MuseGlimmerForConditionalGeneration.forward = muse_glimmer_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_muse_glimmer.MuseGlimmerRMSNorm = LigerRMSNormForMuseGlimmer</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_muse_glimmer.MuseGlimmerTextCenteredRMSNorm = LigerRMSNormForMuseGlimmerTextCentered</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_muse_glimmer.MuseGlimmerTextMLP = LigerSwiGLUMLPForMuseGlimmer</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_muse_glimmer.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_nemotron.CrossEntropyLoss = LigerCrossEntropyLoss</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_nemotron.NemotronForCausalLM.forward = nemotron_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_olmo2.Olmo2ForCausalLM.forward = olmo2_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_olmo2.Olmo2MLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_olmo2.Olmo2RMSNorm = LigerRMSNormForOlmo2</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_olmo2.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_olmo3.Olmo3ForCausalLM.forward = olmo3_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_olmo3.Olmo3MLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_olmo3.Olmo3RMSNorm = LigerRMSNormForOlmo2 # same as olmo2</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_olmo3.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_paligemma.PaliGemmaForConditionalGeneration.forward = lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_paligemma.nn.CrossEntropyLoss = LigerCrossEntropyLoss</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_phi3.Phi3ForCausalLM.forward = phi3_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_phi3.Phi3MLP = LigerPhi3SwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_phi3.Phi3RMSNorm = LigerRMSNorm # Same as Llama</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_phi3.apply_rotary_pos_emb = liger_rotary_pos_emb # Same as Gemma</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_pixtral.PixtralMLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_pixtral.PixtralRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_pixtral.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2.Qwen2ForCausalLM.forward = qwen2_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2.Qwen2MLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2.Qwen2RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_5_vl.CrossEntropyLoss = LigerCrossEntropyLoss</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_5_vl.Qwen2MLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_5_vl.Qwen2RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_5_vl.Qwen2_5_VLForConditionalGeneration.forward = qwen2_5_vl_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_5_vl.Qwen2_5_VLRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_5_vl.apply_multimodal_rotary_pos_emb = liger_multimodal_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_vl.CrossEntropyLoss = LigerCrossEntropyLoss</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_vl.LayerNorm = LigerLayerNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_vl.Qwen2MLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_vl.Qwen2RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_vl.Qwen2VLForConditionalGeneration.forward = qwen2_vl_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_vl.Qwen2VLRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen2_vl.apply_multimodal_rotary_pos_emb = liger_multimodal_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3.Qwen3ForCausalLM.forward = qwen3_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3.Qwen3MLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3.Qwen3RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5.Qwen3_5ForCausalLM.forward = qwen3_5_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5.Qwen3_5ForConditionalGeneration.forward = qwen3_5_lce_forward_for_multimodal</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5.Qwen3_5MLP = LigerQwen3MoeSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5.Qwen3_5RMSNorm = LigerRMSNormForQwen3Next</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5.apply_rotary_pos_emb_vision = liger_rotary_pos_emb_vision</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5_moe.Qwen3_5MoeExperts = LigerExperts</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5_moe.Qwen3_5MoeForCausalLM.forward = qwen3_5_moe_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5_moe.Qwen3_5MoeForConditionalGeneration.forward = (</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5_moe.Qwen3_5MoeRMSNorm = LigerRMSNormForQwen3Next</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5_moe.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_5_moe.apply_rotary_pos_emb_vision = liger_rotary_pos_emb_vision</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_moe.Qwen3MoeExperts = LigerExperts</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_moe.Qwen3MoeForCausalLM.forward = qwen3_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_moe.Qwen3MoeRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_moe.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_next.Qwen3NextExperts = LigerExperts</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_next.Qwen3NextForCausalLM.forward = qwen3_next_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_next.Qwen3NextRMSNorm = LigerRMSNormForQwen3Next</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_next.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_vl.Qwen3VLForConditionalGeneration.forward = qwen3_vl_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_vl.Qwen3VLTextRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_vl.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_vl.apply_rotary_pos_emb_vision = liger_rotary_pos_emb_vision</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_vl_moe.Qwen3VLMoeForConditionalGeneration.forward = qwen3_vl_moe_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_vl_moe.Qwen3VLMoeTextExperts = LigerExperts</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_vl_moe.Qwen3VLMoeTextRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_vl_moe.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_qwen3_vl_moe.apply_rotary_pos_emb_vision = liger_rotary_pos_emb_vision</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_siglip.nn.LayerNorm = LigerLayerNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_smollm3.SmolLM3ForCausalLM.forward = smollm3_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_smollm3.SmolLM3MLP = LigerSwiGLUMLP</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_smollm3.SmolLM3RMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_smollm3.apply_rotary_pos_emb = liger_rotary_pos_emb</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_smolvlm.SmolVLMForConditionalGeneration.forward = smolvlm_lce_forward</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_smolvlm.SmolVLMRMSNorm = LigerRMSNorm</code></td></tr><tr><td><a href="https://github.com/linkedin/Liger-Kernel/blob/HEAD/src/liger_kernel/transformers/monkey_patch.py">linkedin/Liger-Kernel/src/liger_kernel/transformers/monkey_patch.py</a></td><td><code>modeling_smolvlm.nn.LayerNorm = LigerLayerNorm</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/megatron/utils/convert_utils.py">modelscope/ms-swift/swift/megatron/utils/convert_utils.py</a></td><td><code>TEDotProductAttention.forward = forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/megatron/utils/convert_utils.py">modelscope/ms-swift/swift/megatron/utils/convert_utils.py</a></td><td><code>TEDotProductAttention.forward = new_forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/deepseek.py">modelscope/ms-swift/swift/model/models/deepseek.py</a></td><td><code>UnlimitedOCRModel._swift_multi_gpu_patched = True</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/deepseek.py">modelscope/ms-swift/swift/model/models/deepseek.py</a></td><td><code>UnlimitedOCRModel.forward = _patched_forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/deepseek.py">modelscope/ms-swift/swift/model/models/deepseek.py</a></td><td><code>modeling_module.torch.cat = _orig_cat</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/deepseek.py">modelscope/ms-swift/swift/model/models/deepseek.py</a></td><td><code>modeling_module.torch.cat = _safe_cat</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/deepseek.py">modelscope/ms-swift/swift/model/models/deepseek.py</a></td><td><code>transformers.models.llama.modeling_llama.LlamaFlashAttention2 = None</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/internlm.py">modelscope/ms-swift/swift/model/models/internlm.py</a></td><td><code>PreTrainedModel._old_enable_input_require_grads = old_enable_input_require_grads</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/internlm.py">modelscope/ms-swift/swift/model/models/internlm.py</a></td><td><code>PreTrainedModel.enable_input_require_grads = patched_enable_input_require_grads</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/llava.py">modelscope/ms-swift/swift/model/models/llava.py</a></td><td><code>LlavaLlamaForCausalLM.__old_forward = forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/llava.py">modelscope/ms-swift/swift/model/models/llava.py</a></td><td><code>LlavaLlamaForCausalLM.forward = _new_forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/spark.py">modelscope/ms-swift/swift/model/models/spark.py</a></td><td><code>modeling_module._swift_patched = True</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/spark.py">modelscope/ms-swift/swift/model/models/spark.py</a></td><td><code>setattr(modeling_module, name, Spark2_5Loader._compat_mask_fn(getattr(modeling_module, name)))</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/valley.py">modelscope/ms-swift/swift/model/models/valley.py</a></td><td><code>ValleyQwen2ForCausalLM._origin_forward = forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/valley.py">modelscope/ms-swift/swift/model/models/valley.py</a></td><td><code>ValleyQwen2ForCausalLM.forward = new_forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/xing4_0.py">modelscope/ms-swift/swift/model/models/xing4_0.py</a></td><td><code>modeling_module.Xing4_0HyperConnection.forward = torch.compile(</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/xing4_0.py">modelscope/ms-swift/swift/model/models/xing4_0.py</a></td><td><code>modeling_module.Xing4_0PreTrainedModel._can_set_experts_implementation_cached_value = True</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/models/xing4_0.py">modelscope/ms-swift/swift/model/models/xing4_0.py</a></td><td><code>modeling_module._swift_patch_options = patch_options</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/patcher.py">modelscope/ms-swift/swift/model/patcher.py</a></td><td><code>PreTrainedModel._get_no_split_modules = _get_no_split_modules</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/patcher.py">modelscope/ms-swift/swift/model/patcher.py</a></td><td><code>PreTrainedModel._get_no_split_modules = _new_get_no_split_modules</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/patcher.py">modelscope/ms-swift/swift/model/patcher.py</a></td><td><code>PreTrainedModel.from_pretrained = _new_from_pretrained</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/patcher.py">modelscope/ms-swift/swift/model/patcher.py</a></td><td><code>PreTrainedModel.from_pretrained = classmethod(from_pretrained)</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/patcher.py">modelscope/ms-swift/swift/model/patcher.py</a></td><td><code>transformers.modeling_utils.get_balanced_memory = lambda *args, **kwargs: {}</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/model/patcher.py">modelscope/ms-swift/swift/model/patcher.py</a></td><td><code>transformers.modeling_utils.infer_auto_device_map = _infer_auto_device_map_patch</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/sequence_parallel/sequence_parallel.py">modelscope/ms-swift/swift/sequence_parallel/sequence_parallel.py</a></td><td><code>modeling_flash_attention_utils._flash_attention_forward = flash_attention_forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/sequence_parallel/sequence_parallel.py">modelscope/ms-swift/swift/sequence_parallel/sequence_parallel.py</a></td><td><code>modeling_flash_attention_utils._flash_attention_forward_origin = _flash_attention_forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/template/base.py">modelscope/ms-swift/swift/template/base.py</a></td><td><code>modeling_module._flash_attention_forward = _flash_attention_forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/template/base.py">modelscope/ms-swift/swift/template/base.py</a></td><td><code>modeling_module._flash_attention_forward = _origin_flash_attention_forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/template/templates/glm.py">modelscope/ms-swift/swift/template/templates/glm.py</a></td><td><code>modeling_module.create_causal_mask = new_create_causal_mask</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/trainers/mixin.py">modelscope/ms-swift/swift/trainers/mixin.py</a></td><td><code>transformers.modeling_utils.checkpoint = _new_checkpoint</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>AdaLoraModel._create_and_replace = _create_and_replace_hook</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>AdaLoraModel._create_and_replace_origin = AdaLoraModel._create_and_replace</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>AdaLoraModel.forward = adalora_forward</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>BOFTModel._create_and_replace = _create_and_replace_hook</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>BOFTModel._create_and_replace_origin = BOFTModel._create_and_replace</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>BoneModel._create_and_replace = _create_and_replace_hook</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>BoneModel._create_and_replace_origin = BoneModel._create_and_replace</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>FourierFTModel._create_and_replace = _create_and_replace_hook</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>FourierFTModel._create_and_replace_origin = FourierFTModel._create_and_replace</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>LoraModel.__init__ = __new_init__</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>LoraModel.__init_origin__ = LoraModel.__init__</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>LoraModel._cast_adapter_dtype = _cast_adapter_dtype_hook</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>LoraModel._cast_adapter_dtype_origin = LoraModel._cast_adapter_dtype</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>LoraModel._create_and_replace = _create_and_replace_hook</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>LoraModel._create_and_replace_origin = LoraModel._create_and_replace</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>PeftModel.activate_adapter = PeftModel.set_adapter</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>PeftModel.create_optimizer_param_groups = create_optimizer_param_groups</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>PeftModel.deactivate_adapter = partial(dummy_function, func=&#x27;deactivate_adapter&#x27;)</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>PeftModel.load_adapter = load_adapter</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>PeftModel.load_adapter_origin = PeftModel.load_adapter</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>PeftModel.set_active_adapters = partial(dummy_function, func=&#x27;set_active_adapters&#x27;)</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>VeraModel._create_and_replace = _create_and_replace_hook</code></td></tr><tr><td><a href="https://github.com/modelscope/ms-swift/blob/HEAD/swift/tuners/peft.py">modelscope/ms-swift/swift/tuners/peft.py</a></td><td><code>VeraModel._create_and_replace_origin = VeraModel._create_and_replace</code></td></tr><tr><td><a href="https://github.com/sgl-project/sglang/blob/HEAD/python/sglang/multimodal_gen/runtime/layers/attention/layer.py">sgl-project/sglang/python/sglang/multimodal_gen/runtime/layers/attention/layer.py</a></td><td><code>USPAttention._usp_a2a_stream = torch.get_device_module().Stream()</code></td></tr><tr><td><a href="https://github.com/sgl-project/sglang/blob/HEAD/python/sglang/multimodal_gen/runtime/loader/utils.py">sgl-project/sglang/python/sglang/multimodal_gen/runtime/loader/utils.py</a></td><td><code>PreTrainedModel.post_init = lambda self: None</code></td></tr><tr><td><a href="https://github.com/sgl-project/sglang/blob/HEAD/python/sglang/srt/multimodal/processors/qwen_vl.py">sgl-project/sglang/python/sglang/srt/multimodal/processors/qwen_vl.py</a></td><td><code>transformers.models.qwen2_vl.image_processing_qwen2_vl_fast.Qwen2VLImageProcessorFast._preprocess = fast_preprocess_cpu</code></td></tr><tr><td><a href="https://github.com/sgl-project/sglang/blob/HEAD/python/sglang/srt/utils/hf_transformers_patches.py">sgl-project/sglang/python/sglang/srt/utils/hf_transformers_patches.py</a></td><td><code>modeling_llama.LlamaFlashAttention2 = modeling_llama.LlamaAttention</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/compiler.py">unslothai/unsloth-zoo/unsloth_zoo/compiler.py</a></td><td><code>modeling_file.__UNSLOTH_PATCHED__ = True</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/compiler.py">unslothai/unsloth-zoo/unsloth_zoo/compiler.py</a></td><td><code>modeling_file.__UNSLOTH_SUPPORTS_SDPA__ = False</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/compiler.py">unslothai/unsloth-zoo/unsloth_zoo/compiler.py</a></td><td><code>modeling_file.__UNSLOTH_SUPPORTS_SDPA__ = final_supports_sdpa</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/compiler.py">unslothai/unsloth-zoo/unsloth_zoo/compiler.py</a></td><td><code>modeling_file.unsloth_mhc_projection = unsloth_mhc_projection</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/compiler.py">unslothai/unsloth-zoo/unsloth_zoo/compiler.py</a></td><td><code>modeling_file.unsloth_sinkhorn_knopp = unsloth_sinkhorn_knopp</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/compiler.py">unslothai/unsloth-zoo/unsloth_zoo/compiler.py</a></td><td><code>setattr(modeling_file, key, value)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/empty_model.py">unslothai/unsloth-zoo/unsloth_zoo/empty_model.py</a></td><td><code>BitsAndBytesModelLoader._stack_quantization_states = (</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/empty_model.py">unslothai/unsloth-zoo/unsloth_zoo/empty_model.py</a></td><td><code>Gemma4ForCausalLM.get_expert_mapping = get_expert_mapping</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/empty_model.py">unslothai/unsloth-zoo/unsloth_zoo/empty_model.py</a></td><td><code>SiglipVisionModel._init_weights = SiglipVisionModel._original_initialize_weights</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/empty_model.py">unslothai/unsloth-zoo/unsloth_zoo/empty_model.py</a></td><td><code>SiglipVisionModel._init_weights = _init_weights</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/empty_model.py">unslothai/unsloth-zoo/unsloth_zoo/empty_model.py</a></td><td><code>SiglipVisionModel._original_initialize_weights = SiglipVisionModel._init_weights</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/gradient_checkpointing.py">unslothai/unsloth-zoo/unsloth_zoo/gradient_checkpointing.py</a></td><td><code>transformers.modeling_utils._unsloth_old_checkpoint = current</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/gradient_checkpointing.py">unslothai/unsloth-zoo/unsloth_zoo/gradient_checkpointing.py</a></td><td><code>transformers.modeling_utils.checkpoint = (</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/gradient_checkpointing.py">unslothai/unsloth-zoo/unsloth_zoo/gradient_checkpointing.py</a></td><td><code>transformers.modeling_utils.checkpoint = getattr(</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/gradient_checkpointing.py">unslothai/unsloth-zoo/unsloth_zoo/gradient_checkpointing.py</a></td><td><code>transformers.modeling_utils.checkpoint = shim</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/gradient_checkpointing.py">unslothai/unsloth-zoo/unsloth_zoo/gradient_checkpointing.py</a></td><td><code>transformers.modeling_utils.checkpoint = unsloth_checkpoint</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/loss_utils.py">unslothai/unsloth-zoo/unsloth_zoo/loss_utils.py</a></td><td><code>UnslothForCausalLMLoss._unsloth_fused_loss = UnslothForCausalLMLoss</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/loss_utils.py">unslothai/unsloth-zoo/unsloth_zoo/loss_utils.py</a></td><td><code>transformers.modeling_utils.PreTrainedModel.loss_function = \</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/mlx/loader.py">unslothai/unsloth-zoo/unsloth_zoo/mlx/loader.py</a></td><td><code>modeling_llama.LlamaFlashAttention2 = modeling_llama.LlamaAttention</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/common.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/common.py</a></td><td><code>setattr(modeling_module, name, value)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/compiled_model_identity.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/compiled_model_identity.py</a></td><td><code>PreTrainedModel.is_custom_code = method</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/deepseek_v3_moe.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/deepseek_v3_moe.py</a></td><td><code>DeepseekV3ForCausalLM.__UNSLOTH_SUPPORTS_RETURN_HIDDEN_STATES__ = True</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/deepseek_v3_moe.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/deepseek_v3_moe.py</a></td><td><code>DeepseekV3ForCausalLM.forward = _patched_causal_lm_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gemma3n.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gemma3n.py</a></td><td><code>Gemma3nTextAttention.forward = forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gemma4.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gemma4.py</a></td><td><code>Gemma4AudioAttention.forward = forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gemma4.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gemma4.py</a></td><td><code>Gemma4TextAttention.forward = _make_gemma4_attention_carrier_forward(Gemma4TextAttention.forward)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gemma4.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gemma4.py</a></td><td><code>LoraModel._create_and_replace = create_and_replace</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>GptOssForCausalLM._unsloth_grpo_patched = True</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>GptOssForCausalLM.forward = _patched_causal_lm_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>GptOssModel.forward = forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>GptOssPreTrainedModel._unsloth_init_weights_fixed = True</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.generation.utils.create_masks_for_generate = wrap(transformers.generation.utils.create_masks_for_generate)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.masking_utils.__patched_causal_mask__ = True</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.masking_utils._old_create_causal_mask = _torch_compile(transformers.masking_utils.create_causal_mask, fullgraph = False, dynamic = True)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.masking_utils._old_create_sliding_window_causal_mask = _torch_compile(transformers.masking_utils.create_sliding_window_causal_mask, fullgraph = False, dynamic = True)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.masking_utils.create_causal_mask = wrap(create_causal_mask)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.masking_utils.create_masks_for_generate = wrap(transformers.masking_utils.create_masks_for_generate)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.masking_utils.create_sliding_window_causal_mask = wrap(create_sliding_window_causal_mask)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.models.gpt_oss.modeling_gpt_oss.GptOssExperts = GptOssExpertsBnb4bit</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.models.gpt_oss.modeling_gpt_oss.GptOssExperts = \</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.models.gpt_oss.modeling_gpt_oss.GptOssExperts.forward = experts_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.models.gpt_oss.modeling_gpt_oss.GptOssTopKRouter = GptOssTopKRouter</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.models.gpt_oss.modeling_gpt_oss.GptOssTopKRouter = \</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.models.gpt_oss.modeling_gpt_oss._original_GptOssExperts = \</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.models.gpt_oss.modeling_gpt_oss._original_GptOssTopKRouter = \</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.models.gpt_oss.modeling_gpt_oss.create_causal_mask = transformers.masking_utils.create_causal_mask</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.models.gpt_oss.modeling_gpt_oss.create_sliding_window_causal_mask = transformers.masking_utils.create_sliding_window_causal_mask</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.quantizers.quantizer_mxfp4.Mxfp4HfQuantizer._lazy_import_kernels = lambda *args, **kwargs: triton_kernels</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.quantizers.quantizer_mxfp4.Mxfp4HfQuantizer.is_trainable = lambda *args, **kwargs: True</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.quantizers.quantizer_mxfp4.is_kernels_available = is_kernels_available</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/gpt_oss.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/gpt_oss.py</a></td><td><code>transformers.quantizers.quantizer_mxfp4.is_kernels_available = lambda *args, **kwargs: False</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/misc.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/misc.py</a></td><td><code>HiggsAudioV2Embeddings.forward = forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/misc.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/misc.py</a></td><td><code>ModernBertModel._update_attention_mask = _update_attention_mask_contiguous</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/misc.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/misc.py</a></td><td><code>PreTrainedModel.gradient_checkpointing_enable = gradient_checkpointing_enable</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/misc.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/misc.py</a></td><td><code>transformers.models.mllama.modeling_mllama.MllamaVisionEncoderLayer = MllamaVisionEncoderLayer</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/misc.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/misc.py</a></td><td><code>transformers.utils.import_utils.is_causal_conv1d_available = lambda: False</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/moe_utils.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/moe_utils.py</a></td><td><code>PeftModel.load_adapter = patched</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/moe_utils.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/moe_utils.py</a></td><td><code>PeftModel.save_pretrained = patched</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/moe_utils_bnb4bit.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/moe_utils_bnb4bit.py</a></td><td><code>modeling_utils.get_model_conversion_mapping = patched_get_model_conversion_mapping</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/mxfp4.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/mxfp4.py</a></td><td><code>PreTrainedModel.save_pretrained = save_pretrained</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/mxfp4.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/mxfp4.py</a></td><td><code>modeling_utils._get_device_map = _get_device_map</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/mxfp4.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/mxfp4.py</a></td><td><code>transformers.integrations.mxfp4.convert_moe_packed_tensors_cpu = convert_moe_packed_tensors_cpu</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/qwen3_5_moe.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/qwen3_5_moe.py</a></td><td><code>transformers.models.qwen3_5_moe.modeling_qwen3_5_moe.Qwen3_5MoeExperts._unsloth_lora_extractor_fn = staticmethod(_qwen3_5_lora_extractor)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/qwen3_moe.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/qwen3_moe.py</a></td><td><code>transformers.models.qwen3_moe.modeling_qwen3_moe.Qwen3MoeExperts._unsloth_lora_extractor_fn = staticmethod(_qwen3_lora_extractor)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/qwen3_next_moe.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/qwen3_next_moe.py</a></td><td><code>transformers.models.qwen3_next.modeling_qwen3_next.Qwen3NextExperts._unsloth_lora_extractor_fn = staticmethod(_qwen3_next_lora_extractor)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/temporary_patches/qwen3_vl_moe.py">unslothai/unsloth-zoo/unsloth_zoo/temporary_patches/qwen3_vl_moe.py</a></td><td><code>transformers.models.qwen3_vl_moe.modeling_qwen3_vl_moe.Qwen3VLMoeTextExperts._unsloth_lora_extractor_fn = staticmethod(_qwen3_vl_lora_extractor)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth-zoo/blob/HEAD/unsloth_zoo/vllm_utils.py">unslothai/unsloth-zoo/unsloth_zoo/vllm_utils.py</a></td><td><code>Qwen3MoeForCausalLM.is_3d_moe_weight = True</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/studio/backend/core/inference/safetensors_pread.py">unslothai/unsloth/studio/backend/core/inference/safetensors_pread.py</a></td><td><code>modeling_utils.safe_open = _original</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/studio/backend/core/inference/safetensors_pread.py">unslothai/unsloth/studio/backend/core/inference/safetensors_pread.py</a></td><td><code>modeling_utils.safe_open = _with_pread(current)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/import_fixes.py">unslothai/unsloth/unsloth/import_fixes.py</a></td><td><code>PreTrainedModel.disable_input_require_grads = _patched_single_disable_input_require_grads</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/import_fixes.py">unslothai/unsloth/unsloth/import_fixes.py</a></td><td><code>PreTrainedModel.enable_input_require_grads = _patched_enable_input_require_grads</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/import_fixes.py">unslothai/unsloth/unsloth/import_fixes.py</a></td><td><code>PreTrainedModel.enable_input_require_grads = _patched_single_enable_input_require_grads</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/import_fixes.py">unslothai/unsloth/unsloth/import_fixes.py</a></td><td><code>PreTrainedModel.post_init = post_init</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/import_fixes.py">unslothai/unsloth/unsloth/import_fixes.py</a></td><td><code>modeling_utils.revert_weight_conversion = revert_weight_conversion</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/kernels/rms_layernorm.py">unslothai/unsloth/unsloth/kernels/rms_layernorm.py</a></td><td><code>transformers.models.llama.modeling_llama.LlamaRMSNorm = LlamaRMSNorm</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/kernels/rms_layernorm.py">unslothai/unsloth/unsloth/kernels/rms_layernorm.py</a></td><td><code>transformers.models.llama.modeling_llama.LlamaRMSNorm = Unsloth_LlamaRMSNorm</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/kernels/rms_layernorm.py">unslothai/unsloth/unsloth/kernels/rms_layernorm.py</a></td><td><code>transformers.models.mllama.modeling_mllama.MllamaTextRMSNorm = MllamaTextRMSNorm</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/kernels/rms_layernorm.py">unslothai/unsloth/unsloth/kernels/rms_layernorm.py</a></td><td><code>transformers.models.mllama.modeling_mllama.MllamaTextRMSNorm = Unsloth_MllamaTextRMSNorm</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>PreTrainedModel._get_key_renaming_mapping = _get_key_renaming_mapping</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>PreTrainedModel._get_key_renaming_mapping = original</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>PreTrainedModel.gradient_checkpointing_enable = gradient_checkpointing_enable</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>modeling_utils.get_model_conversion_mapping = get_model_conversion_mapping</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>transformers.integrations.bitsandbytes.should_convert_module = patched_should_convert_module</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>transformers.trainer.get_model_param_count = get_model_param_count</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>transformers.trainer_pt_utils.get_model_param_count = get_model_param_count</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>transformers.utils.import_utils.is_flash_attn_2_available = (</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>transformers.utils.import_utils.is_flash_attn_2_available = lambda *args, **kwargs: (</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>transformers.utils.is_flash_attn_2_available = lambda *args, **kwargs: False</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>transformers.utils.is_openai_available = _lazy_openai_import_check(</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/_utils.py">unslothai/unsloth/unsloth/models/_utils.py</a></td><td><code>transformers.utils.quantization_config.BitsAndBytesConfig.__init__ = _BitsAndBytesConfig__init__</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/cohere.py">unslothai/unsloth/unsloth/models/cohere.py</a></td><td><code>CohereAttention.__init__ = eval(init_name)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/cohere.py">unslothai/unsloth/unsloth/models/cohere.py</a></td><td><code>CohereAttention.forward = CohereAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/cohere.py">unslothai/unsloth/unsloth/models/cohere.py</a></td><td><code>CohereDecoderLayer.forward = CohereDecoderLayer_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/cohere.py">unslothai/unsloth/unsloth/models/cohere.py</a></td><td><code>CohereFlashAttention2.forward = CohereAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/cohere.py">unslothai/unsloth/unsloth/models/cohere.py</a></td><td><code>CohereForCausalLM.forward = CausalLM_fast_forward(CohereModel_fast_forward_inference)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/cohere.py">unslothai/unsloth/unsloth/models/cohere.py</a></td><td><code>CohereModel.forward = LlamaModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/cohere.py">unslothai/unsloth/unsloth/models/cohere.py</a></td><td><code>CohereSdpaAttention.forward = CohereAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/cohere.py">unslothai/unsloth/unsloth/models/cohere.py</a></td><td><code>PeftModelForCausalLM.forward = PeftModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/cohere.py">unslothai/unsloth/unsloth/models/cohere.py</a></td><td><code>transformers.models.cohere.modeling_cohere.CohereRotaryEmbedding = LlamaRotaryEmbedding</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/compressed_tensors_int4.py">unslothai/unsloth/unsloth/models/compressed_tensors_int4.py</a></td><td><code>PeftModel.save_pretrained = save_pretrained</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/falcon_h1.py">unslothai/unsloth/unsloth/models/falcon_h1.py</a></td><td><code>FalconH1Attention.__init__ = eval(init_name)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/falcon_h1.py">unslothai/unsloth/unsloth/models/falcon_h1.py</a></td><td><code>FalconH1Attention.forward = FalconH1Attention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/falcon_h1.py">unslothai/unsloth/unsloth/models/falcon_h1.py</a></td><td><code>FalconH1DecoderLayer.forward = FalconH1DecoderLayer_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/falcon_h1.py">unslothai/unsloth/unsloth/models/falcon_h1.py</a></td><td><code>FalconH1ForCausalLM.forward = CausalLM_fast_forward(</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/falcon_h1.py">unslothai/unsloth/unsloth/models/falcon_h1.py</a></td><td><code>FalconH1Model.forward = LlamaModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/falcon_h1.py">unslothai/unsloth/unsloth/models/falcon_h1.py</a></td><td><code>PeftModelForCausalLM.forward = PeftModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/falcon_h1.py">unslothai/unsloth/unsloth/models/falcon_h1.py</a></td><td><code>transformers.models.falcon_h1.modeling_falcon_h1.FalconH1RMSNorm = Unsloth_FalconH1RMSNorm</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/falcon_h1.py">unslothai/unsloth/unsloth/models/falcon_h1.py</a></td><td><code>transformers.models.falcon_h1.modeling_falcon_h1.FalconH1RotaryEmbedding = (</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma.py">unslothai/unsloth/unsloth/models/gemma.py</a></td><td><code>GemmaAttention.__init__ = eval(init_name)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma.py">unslothai/unsloth/unsloth/models/gemma.py</a></td><td><code>GemmaAttention.forward = LlamaAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma.py">unslothai/unsloth/unsloth/models/gemma.py</a></td><td><code>GemmaDecoderLayer.forward = GemmaDecoderLayer_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma.py">unslothai/unsloth/unsloth/models/gemma.py</a></td><td><code>GemmaFlashAttention2.forward = LlamaAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma.py">unslothai/unsloth/unsloth/models/gemma.py</a></td><td><code>GemmaForCausalLM.forward = CausalLM_fast_forward(GemmaModel_fast_forward_inference)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma.py">unslothai/unsloth/unsloth/models/gemma.py</a></td><td><code>GemmaModel.forward = LlamaModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma.py">unslothai/unsloth/unsloth/models/gemma.py</a></td><td><code>GemmaSdpaAttention.forward = LlamaAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma.py">unslothai/unsloth/unsloth/models/gemma.py</a></td><td><code>PeftModelForCausalLM.forward = PeftModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma.py">unslothai/unsloth/unsloth/models/gemma.py</a></td><td><code>transformers.models.gemma.modeling_gemma.GemmaRotaryEmbedding = GemmaFixedRotaryEmbedding</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma2.py">unslothai/unsloth/unsloth/models/gemma2.py</a></td><td><code>Gemma2Attention.__init__ = eval(init_name)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma2.py">unslothai/unsloth/unsloth/models/gemma2.py</a></td><td><code>Gemma2Attention.forward = Gemma2Attention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma2.py">unslothai/unsloth/unsloth/models/gemma2.py</a></td><td><code>Gemma2DecoderLayer.forward = Gemma2DecoderLayer_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma2.py">unslothai/unsloth/unsloth/models/gemma2.py</a></td><td><code>Gemma2FlashAttention2.forward = Gemma2Attention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma2.py">unslothai/unsloth/unsloth/models/gemma2.py</a></td><td><code>Gemma2ForCausalLM.forward = CausalLM_fast_forward(Gemma2Model_fast_forward_inference)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma2.py">unslothai/unsloth/unsloth/models/gemma2.py</a></td><td><code>Gemma2Model.forward = LlamaModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma2.py">unslothai/unsloth/unsloth/models/gemma2.py</a></td><td><code>Gemma2SdpaAttention.forward = Gemma2Attention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma2.py">unslothai/unsloth/unsloth/models/gemma2.py</a></td><td><code>PeftModelForCausalLM.forward = PeftModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/gemma2.py">unslothai/unsloth/unsloth/models/gemma2.py</a></td><td><code>transformers.models.gemma2.modeling_gemma2.Gemma2RotaryEmbedding = GemmaFixedRotaryEmbedding</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/granite.py">unslothai/unsloth/unsloth/models/granite.py</a></td><td><code>GraniteAttention.__init__ = eval(init_name)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/granite.py">unslothai/unsloth/unsloth/models/granite.py</a></td><td><code>GraniteAttention.forward = GraniteAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/granite.py">unslothai/unsloth/unsloth/models/granite.py</a></td><td><code>GraniteDecoderLayer.forward = GraniteDecoderLayer_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/granite.py">unslothai/unsloth/unsloth/models/granite.py</a></td><td><code>GraniteFlashAttention2.forward = GraniteAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/granite.py">unslothai/unsloth/unsloth/models/granite.py</a></td><td><code>GraniteForCausalLM.__init__ = patched_init(GraniteForCausalLM.__init__)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/granite.py">unslothai/unsloth/unsloth/models/granite.py</a></td><td><code>GraniteForCausalLM.forward = CausalLM_fast_forward(GraniteModel_fast_forward_inference)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/granite.py">unslothai/unsloth/unsloth/models/granite.py</a></td><td><code>GraniteModel.forward = LlamaModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/granite.py">unslothai/unsloth/unsloth/models/granite.py</a></td><td><code>GraniteSdpaAttention.forward = GraniteAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/granite.py">unslothai/unsloth/unsloth/models/granite.py</a></td><td><code>PeftModelForCausalLM.forward = PeftModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/granite.py">unslothai/unsloth/unsloth/models/granite.py</a></td><td><code>transformers.models.granite.modeling_granite.GraniteRotaryEmbedding = GraniteRotaryEmbedding</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>FastLlamaModel.from_pretrained = staticmethod(</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>FastLlamaModel.get_peft_model = staticmethod(</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>LlamaAttention.__init__ = eval(init_name)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>LlamaAttention.forward = LlamaAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>LlamaDecoderLayer.forward = LlamaDecoderLayer_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>LlamaFlashAttention2.forward = LlamaAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>LlamaForCausalLM.forward = CausalLM_fast_forward(LlamaModel_fast_forward_inference)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>LlamaModel.forward = LlamaModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>LlamaModel_fast_forward_inference_custom._unsloth_graph_attention = (</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>LlamaSdpaAttention.forward = LlamaAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>PeftModelForCausalLM.forward = PeftModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>transformers.models.llama.modeling_llama.LlamaLinearScalingRotaryEmbedding = (</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/llama.py">unslothai/unsloth/unsloth/models/llama.py</a></td><td><code>transformers.models.llama.modeling_llama.LlamaRotaryEmbedding = LlamaRotaryEmbedding</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/loader.py">unslothai/unsloth/unsloth/models/loader.py</a></td><td><code>PreTrainedModel.from_pretrained = classmethod(wrapped)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/mistral.py">unslothai/unsloth/unsloth/models/mistral.py</a></td><td><code>MistralAttention.__init__ = eval(init_name)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/mistral.py">unslothai/unsloth/unsloth/models/mistral.py</a></td><td><code>MistralAttention.forward = MistralAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/mistral.py">unslothai/unsloth/unsloth/models/mistral.py</a></td><td><code>MistralDecoderLayer.forward = LlamaDecoderLayer_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/mistral.py">unslothai/unsloth/unsloth/models/mistral.py</a></td><td><code>MistralFlashAttention2.forward = MistralAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/mistral.py">unslothai/unsloth/unsloth/models/mistral.py</a></td><td><code>MistralForCausalLM.forward = MistralForCausalLM_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/mistral.py">unslothai/unsloth/unsloth/models/mistral.py</a></td><td><code>MistralModel.forward = LlamaModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/mistral.py">unslothai/unsloth/unsloth/models/mistral.py</a></td><td><code>MistralSdpaAttention.forward = MistralAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/mistral.py">unslothai/unsloth/unsloth/models/mistral.py</a></td><td><code>PeftModelForCausalLM.forward = PeftModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/mistral.py">unslothai/unsloth/unsloth/models/mistral.py</a></td><td><code>transformers.models.mistral.modeling_mistral.MistralRotaryEmbedding = LlamaRotaryEmbedding</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen2.py">unslothai/unsloth/unsloth/models/qwen2.py</a></td><td><code>PeftModelForCausalLM.forward = PeftModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen2.py">unslothai/unsloth/unsloth/models/qwen2.py</a></td><td><code>Qwen2Attention.__init__ = eval(init_name)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen2.py">unslothai/unsloth/unsloth/models/qwen2.py</a></td><td><code>Qwen2Attention.forward = LlamaAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen2.py">unslothai/unsloth/unsloth/models/qwen2.py</a></td><td><code>Qwen2DecoderLayer.forward = LlamaDecoderLayer_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen2.py">unslothai/unsloth/unsloth/models/qwen2.py</a></td><td><code>Qwen2FlashAttention2.forward = LlamaAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen2.py">unslothai/unsloth/unsloth/models/qwen2.py</a></td><td><code>Qwen2ForCausalLM.forward = CausalLM_fast_forward(LlamaModel_fast_forward_inference)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen2.py">unslothai/unsloth/unsloth/models/qwen2.py</a></td><td><code>Qwen2Model.forward = LlamaModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen2.py">unslothai/unsloth/unsloth/models/qwen2.py</a></td><td><code>Qwen2SdpaAttention.forward = LlamaAttention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen2.py">unslothai/unsloth/unsloth/models/qwen2.py</a></td><td><code>transformers.models.qwen2.modeling_qwen2.Qwen2RotaryEmbedding = LlamaRotaryEmbedding</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3.py">unslothai/unsloth/unsloth/models/qwen3.py</a></td><td><code>PeftModelForCausalLM.forward = PeftModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3.py">unslothai/unsloth/unsloth/models/qwen3.py</a></td><td><code>Qwen3Attention.__init__ = eval(init_name)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3.py">unslothai/unsloth/unsloth/models/qwen3.py</a></td><td><code>Qwen3Attention.forward = Qwen3Attention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3.py">unslothai/unsloth/unsloth/models/qwen3.py</a></td><td><code>Qwen3DecoderLayer.forward = LlamaDecoderLayer_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3.py">unslothai/unsloth/unsloth/models/qwen3.py</a></td><td><code>Qwen3FlashAttention2.forward = Qwen3Attention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3.py">unslothai/unsloth/unsloth/models/qwen3.py</a></td><td><code>Qwen3ForCausalLM.forward = CausalLM_fast_forward(</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3.py">unslothai/unsloth/unsloth/models/qwen3.py</a></td><td><code>Qwen3Model.forward = LlamaModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3.py">unslothai/unsloth/unsloth/models/qwen3.py</a></td><td><code>Qwen3SdpaAttention.forward = Qwen3Attention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3.py">unslothai/unsloth/unsloth/models/qwen3.py</a></td><td><code>transformers.models.qwen3.modeling_qwen3.Qwen3RotaryEmbedding = LlamaRotaryEmbedding</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3_moe.py">unslothai/unsloth/unsloth/models/qwen3_moe.py</a></td><td><code>PeftModelForCausalLM.forward = PeftModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3_moe.py">unslothai/unsloth/unsloth/models/qwen3_moe.py</a></td><td><code>Qwen3MoeAttention.__init__ = eval(init_name)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3_moe.py">unslothai/unsloth/unsloth/models/qwen3_moe.py</a></td><td><code>Qwen3MoeAttention.forward = Qwen3Attention_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3_moe.py">unslothai/unsloth/unsloth/models/qwen3_moe.py</a></td><td><code>Qwen3MoeDecoderLayer.forward = Qwen3MoeDecoderLayer_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3_moe.py">unslothai/unsloth/unsloth/models/qwen3_moe.py</a></td><td><code>Qwen3MoeForCausalLM.forward = CausalLM_fast_forward(LlamaModel_fast_forward_inference)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3_moe.py">unslothai/unsloth/unsloth/models/qwen3_moe.py</a></td><td><code>Qwen3MoeMLP.forward = fast_swiglu_inference # This is analogous to Dense models&#x27; MLP</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3_moe.py">unslothai/unsloth/unsloth/models/qwen3_moe.py</a></td><td><code>Qwen3MoeModel.forward = LlamaModel_fast_forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/qwen3_moe.py">unslothai/unsloth/unsloth/models/qwen3_moe.py</a></td><td><code>transformers.models.qwen3_moe.modeling_qwen3_moe.Qwen3MoeRotaryEmbedding = (</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/sentence_transformer.py">unslothai/unsloth/unsloth/models/sentence_transformer.py</a></td><td><code>AutoModel.from_pretrained = original_model_from_pretrained</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/sentence_transformer.py">unslothai/unsloth/unsloth/models/sentence_transformer.py</a></td><td><code>AutoModel.from_pretrained = return_existing_model</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/sentence_transformer.py">unslothai/unsloth/unsloth/models/sentence_transformer.py</a></td><td><code>modeling_distilbert.DistilBertModel.forward = forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/sentence_transformer.py">unslothai/unsloth/unsloth/models/sentence_transformer.py</a></td><td><code>modeling_mpnet.MPNetEncoder.forward = forward</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/sentence_transformer.py">unslothai/unsloth/unsloth/models/sentence_transformer.py</a></td><td><code>modeling_mpnet.MPNetModel._set_gradient_checkpointing = _set_gradient_checkpointing</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/sentence_transformer.py">unslothai/unsloth/unsloth/models/sentence_transformer.py</a></td><td><code>modeling_mpnet.MPNetModel.supports_gradient_checkpointing = True</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/sentence_transformer.py">unslothai/unsloth/unsloth/models/sentence_transformer.py</a></td><td><code>transformers.AutoModelForCausalLM = original_causal</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/sentence_transformer.py">unslothai/unsloth/unsloth/models/sentence_transformer.py</a></td><td><code>transformers.AutoModelForCausalLM = transformers.AutoModel</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/tp_native.py">unslothai/unsloth/unsloth/models/tp_native.py</a></td><td><code>PreTrainedModel.from_pretrained = classmethod(from_pretrained)</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/tp_native.py">unslothai/unsloth/unsloth/models/tp_native.py</a></td><td><code>PreTrainedModel.from_pretrained = descriptor</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/vision.py">unslothai/unsloth/unsloth/models/vision.py</a></td><td><code>FastBaseModel.from_pretrained = staticmethod(</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/vision.py">unslothai/unsloth/unsloth/models/vision.py</a></td><td><code>FastBaseModel.get_peft_model = staticmethod(</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/vision.py">unslothai/unsloth/unsloth/models/vision.py</a></td><td><code>PreTrainedModel.to = original_to</code></td></tr><tr><td><a href="https://github.com/unslothai/unsloth/blob/HEAD/unsloth/models/vision.py">unslothai/unsloth/unsloth/models/vision.py</a></td><td><code>PreTrainedModel.to = to</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/mcore/patch.py">verl-project/verl/verl/models/mcore/patch.py</a></td><td><code>GPTModel._postprocess = _patched</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/mcore/patch.py">verl-project/verl/verl/models/mcore/patch.py</a></td><td><code>MLASelfAttention.get_query_key_value_tensors = patch_get_query_key_value_tensors</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/mcore/patch.py">verl-project/verl/verl/models/mcore/patch.py</a></td><td><code>MultiLatentAttention.forward = patch_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>AutoModelForCausalLMWithValueHead.state_dict = state_dict</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Glm4vModel.forward = glm4v_base_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Glm4vTextAttention.forward = glm4v_attn_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen2VLAttention.forward = qwen2_vl_attn_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen2VLModel.forward = qwen2_vl_base_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen2_5_VLAttention.forward = qwen2_vl_attn_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen2_5_VLModel.forward = qwen2_vl_base_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen3VLModel.forward = qwen3_vl_base_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen3VLMoeModel.forward = qwen3_vl_base_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen3VLMoeVisionModel.fast_pos_embed_interpolate = fast_pos_embed_interpolate</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen3VLVisionModel.fast_pos_embed_interpolate = fast_pos_embed_interpolate</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen3_5DecoderLayer.forward = qwen3_5_decoder_layer_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen3_5Model.forward = qwen3_5_base_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen3_5MoeDecoderLayer.forward = qwen3_5_decoder_layer_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen3_5MoeModel.forward = qwen3_5_base_forward</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen3_5MoeVisionModel.fast_pos_embed_interpolate = fast_pos_embed_interpolate</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/monkey_patch.py">verl-project/verl/verl/models/transformers/monkey_patch.py</a></td><td><code>Qwen3_5VisionModel.fast_pos_embed_interpolate = fast_pos_embed_interpolate</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen2.Qwen2MLP.forward = silu_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen2.Qwen2RMSNorm.forward = rms_norm_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen2.apply_rotary_pos_emb = apply_rotary_pos_emb_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen2_5_vl.Qwen2RMSNorm.forward = rms_norm_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen2_5_vl.Qwen2_5_VLMLP.forward = silu_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen2_5_vl.Qwen2_5_VLRMSNorm.forward = rms_norm_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3.Qwen3MLP.forward = silu_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3.Qwen3RMSNorm.forward = rms_norm_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3.apply_rotary_pos_emb = apply_rotary_pos_emb_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_5.Qwen3_5RMSNorm.forward = qwen3_next_rms_norm_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_5.Qwen3_5RMSNormGated.forward = qwen3_next_rms_norm_forward_gated_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_5.apply_rotary_pos_emb = qwen3_next_apply_rotary_pos_emb_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_5_moe.Qwen3_5MoeExperts.forward = qwen3_5_moe_experts_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_5_moe.Qwen3_5MoeRMSNorm.forward = qwen3_next_rms_norm_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_5_moe.Qwen3_5MoeRMSNormGated.forward = qwen3_next_rms_norm_forward_gated_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_5_moe.apply_rotary_pos_emb = qwen3_next_apply_rotary_pos_emb_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_moe.Qwen3MoeRMSNorm.forward = rms_norm_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_moe.Qwen3MoeSparseMoeBlock.forward = qwen3_moe_sparse_moe_block_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_moe.apply_rotary_pos_emb = apply_rotary_pos_emb_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_next.Qwen3NextRMSNorm.forward = qwen3_next_rms_norm_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_next.Qwen3NextRMSNormGated.forward = qwen3_next_rms_norm_forward_gated_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_next.Qwen3NextSparseMoeBlock.forward = qwen3_next_sparse_moe_block_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_next.apply_rotary_pos_emb = qwen3_next_apply_rotary_pos_emb_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_vl.Qwen3VLTextMLP.forward = silu_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_vl.Qwen3VLTextRMSNorm.forward = rms_norm_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_vl_moe.Qwen3VLMoeTextRMSNorm.forward = rms_norm_forward_npu</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_vl_moe.Qwen3VLMoeTextSparseMoeBlock = NPUQwen3VLMoeTextSparseMoeBlock</code></td></tr><tr><td><a href="https://github.com/verl-project/verl/blob/HEAD/verl/models/transformers/npu_patch.py">verl-project/verl/verl/models/transformers/npu_patch.py</a></td><td><code>modeling_qwen3_vl_moe.apply_rotary_pos_emb = apply_rotary_pos_emb_npu</code></td></tr><tr><td><a href="https://github.com/vllm-project/llm-compressor/blob/HEAD/examples/quantizing_moe/deepseek_v4_example.py">vllm-project/llm-compressor/examples/quantizing_moe/deepseek_v4_example.py</a></td><td><code>DeepseekV4PreTrainedModel._keep_in_fp32_modules_strict = set()</code></td></tr><tr><td><a href="https://github.com/vllm-project/llm-compressor/blob/HEAD/examples/quantizing_moe/deepseek_v4_mixed_mxfp4_mxfp8.py">vllm-project/llm-compressor/examples/quantizing_moe/deepseek_v4_mixed_mxfp4_mxfp8.py</a></td><td><code>DeepseekV4PreTrainedModel._keep_in_fp32_modules_strict = set()</code></td></tr><tr><td><a href="https://github.com/vllm-project/llm-compressor/blob/HEAD/examples/quantizing_moe/deepseek_v4_pro_example.py">vllm-project/llm-compressor/examples/quantizing_moe/deepseek_v4_pro_example.py</a></td><td><code>DeepseekV4PreTrainedModel._keep_in_fp32_modules_strict = set()</code></td></tr><tr><td><a href="https://github.com/vllm-project/llm-compressor/blob/HEAD/src/llmcompressor/modeling/moe/linearize.py">vllm-project/llm-compressor/src/llmcompressor/modeling/moe/linearize.py</a></td><td><code>transformers.integrations.finegrained_fp8.replace_with_fp8_linear = (</code></td></tr><tr><td><a href="https://github.com/vllm-project/vllm/blob/HEAD/vllm/model_executor/models/adapters.py">vllm-project/vllm/vllm/model_executor/models/adapters.py</a></td><td><code>ModelForEmbedding.__name__ = _get_pooling_model_name(cls.__name__, &quot;ForEmbedding&quot;)</code></td></tr><tr><td><a href="https://github.com/vllm-project/vllm/blob/HEAD/vllm/model_executor/models/expert_substitution.py">vllm-project/vllm/vllm/model_executor/models/expert_substitution.py</a></td><td><code>ExpertSubstitutionModel.__name__ = model_cls.__name__</code></td></tr><tr><td><a href="https://github.com/vllm-project/vllm/blob/HEAD/vllm/model_executor/models/expert_substitution.py">vllm-project/vllm/vllm/model_executor/models/expert_substitution.py</a></td><td><code>ExpertSubstitutionModel.__qualname__ = model_cls.__qualname__</code></td></tr><tr><td><a href="https://github.com/vllm-project/vllm/blob/HEAD/vllm/v1/attention/backends/composite.py">vllm-project/vllm/vllm/v1/attention/backends/composite.py</a></td><td><code>CompositeAttentionBackend.__module__ = module</code></td></tr><tr><td><a href="https://github.com/vllm-project/vllm/blob/HEAD/vllm/v1/attention/backends/composite.py">vllm-project/vllm/vllm/v1/attention/backends/composite.py</a></td><td><code>CompositeAttentionBackend.__name__ = name</code></td></tr><tr><td><a href="https://github.com/vllm-project/vllm/blob/HEAD/vllm/v1/attention/backends/composite.py">vllm-project/vllm/vllm/v1/attention/backends/composite.py</a></td><td><code>CompositeAttentionBackend.__qualname__ = name</code></td></tr></table></details>