ludandaye/mistral-7b-collapse-exp6

Mistral-7B Model Collapse Chain (exp6_r_001)

Multi-generational model collapse experiment using Mistral-7B-v0.1.

Experiment Setup

• Base model: mistralai/Mistral-7B-v0.1
• Dataset: OpenWebText (5000 training samples per generation)
• Strategy: Replace (each generation trains on previous generation output only)
• Generations: 0–10
• Seed: 42

Results

Gen	MAUVE	δₖ	PPL	Rep Rate
0	0.868	0.132	14.4	5.9%
1	0.720	0.280	30.7	13.4%
2	0.529	0.471	56.4	17.4%
3	0.413	0.587	106.6	18.2%
4	0.242	0.758	164.5	20.0%
5	0.182	0.818	271.0	20.5%
6	0.125	0.875	563.2	18.7%
7	0.094	0.906	1091.8	20.7%
8	0.074	0.926	1736.5	21.3%
9	0.050	0.950	4643.5	21.2%
10	0.039	0.961	4711.3	20.3%

Key Findings

• MAUVE exponential decay: 0.868 → 0.039 over 10 generations
• PPL explosion: 14.4 → 4711.3 (327× increase)
• Transfer function: Power-law δₙ₊₁ = 0.989·δₙ^0.610 (R²=0.992)
• Repetition rate saturates at ~20%, indicating collapse is distributional shift, not mere repetition

File Structure

Each gen_X/ folder contains the full model checkpoint (config, tokenizer, safetensors).

Citation

Part of the IQD (Information Quality Density) research project studying multi-generational AI model collapse.