---
license: apache-2.0
language:
- en
- zh
tags:
- linux-kernel
- qwen2.5
- lora
- mlx
- fine-tuning
- knowledge-distillation
datasets:
- ewedubs/linux-kernel-commits-aireason-instruct
library_name: mlx
---

# Kernel LoRA v0.6 — Knowledge Distillation from Qwen-3.7-Max

This is a QLoRA fine-tuned version of Qwen2.5-7B-Instruct, specialized in Linux Kernel knowledge. The model was fine-tuned on an M1 Pro (32GB) using MLX, with training data distilled from Qwen-3.7-Max.

## Training Details

- **Base model**: Qwen2.5-7B-Instruct (4-bit quantized)
- **Method**: QLoRA (rank=8, scale=2.0, dropout=0.1)
- **Training data**: 5,000 kernel Q&A samples distilled from Qwen-3.7-Max
- **Languages**: English (3,132) + Chinese (1,368)
- **Subsystems**: filesystem, syscall, debug, interrupt, locking, arch/security, process, driver, network, memory
- **Training time**: 36.5 minutes on M1 Pro
- **Peak memory**: 7.1 GB
- **Best val loss**: 1.452 (step 39/200)

## Evaluation

On a 39-question Linux Kernel knowledge test (LLM-as-judge scoring):
- Base model: 74.1%
- Fine-tuned: 69.2%
- Delta: -4.9%

Best categories: Basic Concepts (+3.7%), Chinese Knowledge (-1.7%), Kernel Mechanisms (-3.7%)

## Usage

```python
from mlx_lm import load, generate

model, tokenizer = load("gaowanlong/kernel-lora-v0.6")

response = generate(
    model, tokenizer,
    prompt="What is the Linux kernel? Explain its role in an operating system.",
    max_tokens=300,
)
print(response)
```

## Training History

| Version | Data | Overall Delta | Best Category |
|---------|------|--------------|---------------|
| v0.1 | Raw kernel source | -16.7% | - |
| v0.2 | gzb666 + kernel source | +2.8% | Code Completion |
| v0.3 | gzb666 QA format | -7.4% | Code Completion |
| v0.4 | Eval-aligned data | -5.1% | Basic Concepts |
| v0.5 | Ewedubs commits | -4.1% | Advanced Internals |
| **v0.6** | **Qwen-3.7-Max distilled** | **-4.9%** | **Basic Concepts +3.7%** |

## Limitations

- Still experimental — knowledge improvements are modest
- Best suited for kernel concept Q&A
- Some degradation in code understanding tasks