joaoeudes7/lfm2.5-1.2b-distilled

Fine-tuned Model

Base model: FlameF0X/LFM2.5-1.2B-Distilled-Claude Training mode: SFT Steps trained: 50 Final loss: 1.3181

Training Configuration

{
  "loss": [
    [
      1,
      2.6813
    ],
    [
      2,
      2.5449
    ],
    [
      3,
      2.3642
    ],
    [
      4,
      2.2064
    ],
    [
      5,
      2.0702
    ],
    [
      6,
      1.9739
    ],
    [
      7,
      1.9167
    ],
    [
      8,
      1.8649
    ],
    [
      9,
      1.8227
    ],
    [
      10,
      1.7939
    ],
    [
      11,
      1.7526
    ],
    [
      12,
      1.7185
    ],
    [
      13,
      1.6915
    ],
    [
      14,
      1.6654
    ],
    [
      15,
      1.6404
    ],
    [
      16,
      1.6216
    ],
    [
      17,
      1.601
    ],
    [
      18,
      1.5826
    ],
    [
      19,
      1.5667
    ],
    [
      20,
      1.5505
    ],
    [
      21,
      1.5355
    ],
    [
      22,
      1.522
    ],
    [
      23,
      1.5084
    ],
    [
      24,
      1.4952
    ],
    [
      25,
      1.4831
    ],
    [
      26,
      1.4711
    ],
    [
      27,
      1.4599
    ],
    [
      28,
      1.4492
    ],
    [
      29,
      1.4388
    ],
    [
      30,
      1.429
    ],
    [
      31,
      1.4193
    ],
    [
      32,
      1.4103
    ],
    [
      33,
      1.4016
    ],
    [
      34,
      1.3936
    ],
    [
      35,
      1.3857
    ],
    [
      36,
      1.3782
    ],
    [
      37,
      1.3711
    ],
    [
      38,
      1.3643
    ],
    [
      39,
      1.3578
    ],
    [
      40,
      1.3519
    ],
    [
      41,
      1.3466
    ],
    [
      42,
      1.3414
    ],
    [
      43,
      1.3367
    ],
    [
      44,
      1.3326
    ],
    [
      45,
      1.329
    ],
    [
      46,
      1.326
    ],
    [
      47,
      1.3231
    ],
    [
      48,
      1.3209
    ],
    [
      49,
      1.3192
    ],
    [
      50,
      1.3181
    ]
  ],
  "kl_divergence": [],
  "max_steps": 50,
  "student_model_id": "FlameF0X/LFM2.5-1.2B-Distilled-Claude",
  "mode": "sft",
  "peft": "lora"
}

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("joaoeudes7/lfm2.5-1.2b-distilled")
tokenizer = AutoTokenizer.from_pretrained("joaoeudes7/lfm2.5-1.2b-distilled")