Praha-Labs/MiniCPM5-2B-TikZ-LoRA

MiniCPM5-2B TikZ LoRA

Stage 1 LoRA supervised fine-tuning for instruction-to-TikZ generation. This flat repository contains the exact pinned BF16 base checkpoint and the unmerged LoRA adapter at repository root, plus tokenizer, metrics, provenance, and checksums.

Provenance

• Base model: openbmb/MiniCPM5-2B
• Base revision: 12a3808a956f869c767195e9266b59c4d21d92e2
• Training method: BF16 LoRA, not QLoRA
• Trainer: Unsloth native loading with TRL SFTTrainer
• Hardware: 1 x NVIDIA A40 48 GB
• Maximum sequence length: 8192
• Trainable parameters observed: 100,466,688
• Total parameters observed: 2,617,223,168
• Trainable fraction: 3.8387%

Dataset and tokens

• Training examples: 95347
• Eligible selected examples: 95347
• Quarantined examples excluded: 1153
• Prompt tokens: 19578751
• Supervised assistant tokens: 56063413
• Unpadded dataset tokens: 75642164
• Trainer-reported input tokens processed: 107296408

Only assistant/TikZ tokens contributed to loss. Prompt tokens were masked and overlength examples were quarantined rather than truncated.

Optimization

• Epochs completed: 1.000000
• Effective batch size: 16
• Per-device batch size: 2
• Gradient accumulation: 8
• Learning rate: 0.0001
• Scheduler: cosine
• LoRA rank: 64
• LoRA alpha: 64
• LoRA dropout: 0.0

Results

• Overall gate passed: True
• Training completed: True
• Full epoch completed: True
• Artifact complete: True
• Validation loss: 0.449503
• Trainer-reported train loss: 0.061229
• First logged loss: 0.589100
• Final logged loss: 0.572400
• Minimum logged loss: 0.363200
• Mean logged loss: 0.537604
• Trainer-reported input tokens: 107296408
• Total FLOPs: 1.512773958542721e+18
• Accumulated Slurm allocation time: 10h 9m 38s
• Training allocations: 6

Loss is token-level cross-entropy against one reference TikZ implementation; it does not directly measure compilation or rendered-image similarity.

Slurm allocations

Job ID	Name	State	Elapsed	Node
4317286	minicpm-native-full	TIMEOUT	01:00:03	a0429
4321521	minicpm-native-full-r2	TIMEOUT	02:00:25	a0429
4322440	minicpm-native-full-r3	TIMEOUT	02:00:17	a0429
4324001	minicpm-native-full-r4	TIMEOUT	02:00:20	a0429
4326352	minicpm-native-full-r2	TIMEOUT	02:00:22	a0429
4327247	minicpm-native-final	COMPLETED	01:08:11	a0226

Loading

The standard Transformers PEFT integration can load the adapter using its recorded base-model reference:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "Praha-Labs/MiniCPM5-2B-TikZ-LoRA"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True
)

Explicit PEFT loading against the pinned upstream base is also supported:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

repo = "Praha-Labs/MiniCPM5-2B-TikZ-LoRA"
base = AutoModelForCausalLM.from_pretrained(
    "openbmb/MiniCPM5-2B", revision="12a3808a956f869c767195e9266b59c4d21d92e2",
    torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(repo)
model = PeftModel.from_pretrained(base, repo)

Included evidence

training-metrics.json, resolved-config.json, training-config.yaml, native-run.json, token-report.json, training-jobs.tsv, environment.lock, source-commit.txt, BASE_MODEL_CARD.md, and SHA256SUMS document the run and its provenance.

Limitations

Compilation rate, rendered-image similarity, and human evaluation should be measured before deployment because multiple valid TikZ programs can render the same diagram.