--- license: apache-2.0 --- # Fragmented Training Enables Surgical Layer Pruning in Causal Language Models via Early Confidence Sharpening **Authors**: aifeifei, Gemini **Affiliation**: Independent Research / Open Source Community **Repository**: https://huggingface.co/aifeifei798/Fragmented-Training --- ## Abstract Large Language Models (LLMs) suffer from significant inference latency during autoregressive generation due to their deep memory-bound architectures. While structural layer pruning offers a direct approach to physical speedup, naive pruning often causes severe semantic collapse due to representation mismatch across unaligned residual streams. In this paper, we introduce **Fragmented Training (FT)**, a novel fine-tuning paradigm that applies stochastic token shuffling (70% noise rate) during Instruction Fine-Tuning (SFT) to decouple logical intent from strict linear syntax. Mechanistic audits via Logit Lens and Residual Stream Vector Interference demonstrate that FT forces the model to perform early decision commitment, driving a dramatic **Active Vocabulary Collapse** ($P > 0.1\%$) from 72 candidates down to 8 candidates in deep layers (a drop of -64 active tokens at Layer 29). Exploiting this induced deep-layer redundancy, we perform **Surgical Layer Pruning** by removing 4 contiguous redundant deep layers (L-29 to L-32, reducing parameter count by 11.1%) and applying a lightweight 1-epoch representation stitching alignment ($h_{28} \to h_{33}$). End-to-end hardware benchmarks on Qwen3-4B using an NVIDIA GeForce RTX 5090 D prove that our pruned model eliminates repetitive generation loops, preserves complex zero-shot reasoning, and achieves a net **+13.15% hardware throughput acceleration** (18.79 to 21.26 Tokens/s). --- ## 1. Introduction Autoregressive Large Language Models (LLMs) have achieved state-of-the-art performance across various reasoning tasks. However, serving deep transformer models in real-world scenarios remains computationally expensive and memory-bound. Each generated token requires passing hidden representations through all $L$ layers, making physical inference latency directly proportional to network depth $L$. Standard layer pruning methods attempt to reduce $L$ by dropping intermediate layers based on heuristic importance scores. However, directly removing layers introduces a severe **Representation Shift** in the residual stream: layer $L_{k+1}$ receives an uncalibrated hidden state $h_k$ rather than $h_{k+m}$, leading to semantic disorientation, token repetition loops, and catastrophic degradation in reasoning capability. ``` [Standard Base LLM] ==> Slow Entropy Collapse (Hesitates across 40~72 candidates till L-33) [FT Model + Prune] ==> Early Confidence Sharpening (L-28 Locks into 7 Tokens) └──> Safe Pruning of L-29~L-32 + 1-Epoch Stitching (TPS +13.15%) ``` In this work, we propose a synergistic two-stage framework: 1. **Fragmented Training (FT)**: We subject input prompts to a 70% stochastic word-shuffling transformation during SFT while keeping target outputs pristine. This "Cognitive Burden" forces the model to develop robust internal denoising, shifting its decision-making forward and inducing **Confidence Sharpening** in deeper layers. 2. **Mechanistic Audit & Surgical Pruning**: Using Logit Lens and Layer-wise Residual Vector Interference $\cos(h_l, \Delta h_l)$, we pinpoint layers L-29 to L-32 as computationally redundant "zero-friction" passages. We hard-prune these 4 layers and execute a 1-epoch Representation Stitching SFT to realign $h_{28}$ with $h_{33}$. --- ## 2. Methodology ### 2.1 Fragmented Training (FT) Paradigm Formally, let $X = (x_1, x_2, \dots, x_N)$ denote the sequence of input prompt tokens and $Y = (y_1, y_2, \dots, y_M)$ denote the ground-truth target tokens. During standard SFT, the objective minimizes $-\sum \log P(y_t \mid X, y_{ 3: num_to_shuffle = int(len(words) * burden_ratio) indices = random.sample(range(len(words)), num_to_shuffle) shuffled_subset = [words[i] for i in indices] random.shuffle(shuffled_subset) for idx, orig_i in enumerate(indices): words[orig_i] = shuffled_subset[idx] return ' '.join(words) return text ``` ### 2.2 Mechanistic Audit Framework To diagnose internal representations without modifying model weights, we employ two diagnostic metrics: 1. **Active Vocabulary Count ($P > 0.1\%$)**: We project the normalized hidden state $h_l$ at layer $l$ through the final unembedding head: $$ \mathbf{z}_l = \text{LM\_Head}(\text{Final\_Norm}(h_l)) $$ $$ \mathbf{p}_l = \text{Softmax}(\mathbf{z}_l) $$ $$ N_{\text{active}}^{(l)} = \sum_{v \in \mathcal{V}} \mathbb{I}(p_{l, v} > 0.001) $$ 2. **Residual Stream Vector Interference**: We track the residual update $\Delta h_l = h_{l+1} - h_l$ and compute its cosine similarity with the accumulated stream $h_l$: $$ \text{Sim}^{(l)} = \cos(h_l, \Delta h_l) = \frac{h_l \cdot \Delta h_l}{\|h_l\|_2 \|\Delta h_l\|_2} $$ ### 2.3 Surgical Layer Pruning & Representation Stitching Based on the active token audit, we remove $M=4$ contiguous layers (L-29 to L-32). To resolve the resulting representation mismatch ($h_{28} \to h_{33}$), we execute a lightweight **Stitching Alignment**: * **Dataset**: 100–200 clean, uncorrupted instruction samples ($\gamma = 0$). * **Optimization**: Train a LoRA adapter on $W_Q, W_K, W_V, W_O, W_{\text{gate}}, W_{\text{up}}, W_{\text{down}}$ for 1 epoch (100 steps, learning rate $\eta = 10^{-4}$). --- ## 3. Empirical Experiments & Diagnostics ### 3.1 Active Vocabulary Collapse We evaluate Qwen3-4B (36 layers) before and after FT training. Table 1 summarizes the Active Candidate Tokens ($P > 0.1\%$) across deep reasoning layers. **Table 1**: Active Vocabulary Count Comparison across Deep Layers. | Layer $l$ | Base Model ($P > 0.1\%$) | FT Model ($P > 0.1\%$) | Difference ($\text{FT} - \text{Base}$) | Dynamics | | :--- | :---: | :---: | :---: | :--- | | **L-19** | 66 | 72 | +6 | Candidate Assembly (Broad Search) | | **L-22** | 99 | 73 | -26 | Pruning Onset | | **L-23** | 82 | 47 | -35 | Subtractive Pruning | | **L-27** | 57 | 12 | -45 | Severe Vocabulary Collapse | | **L-28** | 43 | **7** | -36 | Single-Digit Lock (7 Tokens) | | **L-29** | 72 | **8** | **-64** | **Peak Collapse (Pruning Cliff)** | | **L-30** | 52 | 15 | -37 | Sustained Compression | | **L-31** | 36 | **6** | -30 | Single-Digit Lock (6 Tokens) | | **L-32** | 41 | **7** | -34 | Single-Digit Lock (7 Tokens) | | **L-33** | 15 | **7** | -8 | Ready for Sharp Output | As shown in Table 1, the Base model hesitates across 72 candidate words at Layer 29. In contrast, the FT model aggressively prunes off-target branches down to 8 tokens at Layer 29, achieving single-digit candidate lock 5 layers earlier than the Base model. --- ### 3.2 Residual Interference Dynamics Vector interference metrics reveal a distinct 3-phase denoising mechanism in the FT model: **Table 2**: Layer-wise Cosine Interference $\cos(h_l, \Delta h_l)$. | Layer $l$ | Base $\cos(h_l, \Delta h_l)$ | FT $\cos(h_l, \Delta h_l)$ | Delta $\Delta \cos$ | Tensor Interference Phase | | :--- | :---: | :---: | :---: | :--- | | **L-10** | -0.3186 | -0.3297 | -0.0111 | Phase 1: Amplified Noise Subtraction | | **L-12** | -0.1229 | -0.1680 | **-0.0452** | Phase 1: Peak Destructive Denoising | | **L-19** | +0.1779 | +0.0425 | **-0.1353** | Phase 2: Massive Noise Path Suppression | | **L-33** | +0.4352 | +0.5018 | **+0.0665** | Phase 3: Frictionless Target Push | 1. **Phase 1 (L-10 to L-14)**: Amplified destructive interference ($\cos < 0$) actively cancels out noise introduced by shuffled tokens. 2. **Phase 2 (L-19)**: Off-target positive update paths are suppressed ($\Delta \cos = -0.1353$), preventing hallucination propagation. 3. **Phase 3 (L-23 to L-33)**: Deep layers show frictionless positive alignment ($\cos \to 0.5018$), enabling target convergence. --- ### 3.3 End-to-End Hardware Speedup Benchmark We evaluate physical inference performance on an **NVIDIA GeForce RTX 5090 D GPU**. We compare the Original 36-layer Base Model against our Exported 32-layer Pruned & Stitched Standalone Model under identical generation settings (`max_new_tokens=256`, `temperature=0.7`, `top_p=0.8`, `do_sample=True`). **Table 3**: End-to-End Hardware Performance Comparison. | Model Architecture | Layers | Weights Load Time | Total Latency (256 Tokens) | Throughput (Tokens/s) | Net TPS Speedup | | :--- | :---: | :---: | :---: | :---: | :---: | | **Original Base Model** | 36 | 6.12 s | 13.6256 s | 18.79 Tokens/s | Baseline | | **Final Pruned Standalone** | **32** | **4.08 s** | **12.0423 s** | **21.26 Tokens/s** | **+13.15%** | **Key Findings**: * **Physical Acceleration**: Throughput increases from 18.79 Tokens/s to 21.26 Tokens/s, a net **+13.15% speedup**, closely matching the theoretical 11.11% layer reduction. * **Repetition Loop Fix**: Unstitched pruned models collapsed into infinite repetition loops (13.03 s for garbage tokens). The 1-epoch stitching alignment completely eliminated token loops and restored fluent reasoning. --- ## 4. Discussion & Limitations ### Why 1-Epoch Stitching Succeeds Because Transformer layers communicate via additive residual connections ($h_{l+1} = h_l + f(h_l)$), hidden states across all layers reside in the same $d_{\text{model}}$ vector space. Reconnecting $h_{28}$ directly to $h_{33}$ does not require learning new semantic knowledge; it merely requires recalibrating affine feature scales. In high-dimensional space, gradient updates during 1 epoch (100 steps) are sufficient to achieve smooth feature alignment. ### Limitations Small-scale models (4B) exhibit sensitivity to long-tail entity names when pruned. In the absence of domain-specific recovery data, setting a mild repetition penalty (`repetition_penalty = 1.15 -- 1.20`) or employing Self-Distillation from the unpruned teacher model is recommended to prevent local semantic repetition. --- ## 5. Conclusion We presented a synergistic approach combining **Fragmented Training (FT)**, mechanistic Logit Lens audits, surgical layer pruning, and representation stitching. By inducing early confidence sharpening during instruction tuning, FT enables removing 4 deep redundant layers from Qwen3-4B, yielding a standalone 32-layer model with a **+13.15% physical inference throughput gain** without compromising reasoning capabilities. --- ## Citation ```bibtex @misc{aifeifei_2026, author = { aifeifei and Gemini }, title = { Fragmented-Training: Accelerating LLM Inference via Stochastic Token Shuffling and Surgical Layer Pruning }, year = 2026, url = { https://huggingface.co/aifeifei798/Fragmented-Training }, doi = { 10.57967/hf/7592 }, publisher = { Hugging Face } } ```