README.md

---
library_name: mlx
license: apache-2.0
base_model: LiquidAI/LFM2.5-230M
tags:
  - mlx
  - lora
  - dialogue
  - conversational
language: en
pipeline_tag: text-generation
---

# questions-lfm2-4bit

LFM2.5-230M fine-tuned with **MLX LoRA** to extract the most recent question from a multi-turn dialogue. Input is a transcript tagged with `[S]` and `[M]` line prefixes. The model **prioritizes** questions appearing under the `[S]` tag; if no question is present there, it falls back to the latest `[M]` block.

End-to-end latency on M-series Mac: **100–150 ms** per call (greedy decoding, max 80 tokens, 4-bit quantized).

## Pipeline

1. LoRA fine-tune in MLX (rank 32, 200 iters, 6.19M trainable params / 2.7%)
2. Fuse adapter into base model → fp16
3. Quantize to 4-bit (group size 64)

## Input format

Transcript with `[S]` and `[M]` line prefixes:

```
[S] what would you like to discuss today
[M] i was thinking about the architecture of the new service
[S] ok
[M] could you walk me through the current approach
```

## Output

A single extracted question (no question mark, no quotes). Priority order:

1. Latest question found under `[S]`
2. Otherwise, latest question found under `[M]`
3. Otherwise, the most recent `[M]` content

## Usage (Python)

```python
from mlx_lm import load, generate

model, tokenizer = load("GameGC/questions-lfm2-4bit")

messages = [
    {"role": "system", "content": "Extract the most recent question from the dialogue."},
    {"role": "user", "content": "[S] what would you like to discuss\n[M] i was thinking about the architecture\n[S] ok\n[M] could you walk me through the current approach"},
]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
out = generate(model, tokenizer, prompt=prompt, max_tokens=80)
print(out)
```

## Usage (Swift)

```swift
import MLXLMCommon

let config = ModelConfiguration(id: "GameGC/questions-lfm2-4bit")
```

## Performance

| Metric | Value |
|---|---|
| Latency (M-series Mac, 4-bit) | 100–150 ms per call |
| Model size on disk | ~134 MB |
| Max context | 1024 tokens |
| Quantization | 4-bit, group size 64 |

## Limitations

- Declarative statements under `[M]` containing question-sounding words ("how", rhetorical "right") may trigger false-positive extraction. Will be addressed in v2.
- 4-bit quantization introduces a minor quality regression vs the fp16 fused version.
- Maximum context length: 1024 tokens. Longer transcripts are truncated.

## Training

| Hyperparameter | Value |
|---|---|
| Base | LiquidAI/LFM2.5-230M |
| Method | MLX LoRA |
| Rank | 32 |
| Alpha | 64 |
| LoRA keys | `self_attn.{q,k,v,out}_proj`, `feed_forward.{w1,w2,w3}` |
| Trainable params | 6.19M (2.696%) |
| Iters | 200 |
| Learning rate | 2e-4 (cosine decay, warmup 10) |
| Max seq length | 1024 |
| Batch size | 4 × 4 grad accum |
| Val loss | 3.264 → 0.360 |
| Train loss | 3.144 → 0.368 |
| Duration | 110s on M-series Mac |