SmolLM2-135M-Post-ASR-Engine

A compact, local fine-tune of HuggingFaceTB/SmolLM2-135M-Instruct designed to act as a lightweight post-ASR (Automatic Speech Recognition) formatting utility.

Purpose & Scope

Raw ASR transcripts often lack punctuation, structure, capitalization, and contain disfluencies. This model transforms raw spoken transcripts into clean, structured text using specific task tokens.

Supported Task Tokens

1. <|task:format_email|>: Formats raw spoken notes into a structured email with Subject line and body.
2. <|task:bullet_points|>: Extracts concise, action-oriented bullet points from meeting or standup dictations.
3. <|task:clean_punctuate|>: Restores punctuation, capitalization, numbers, and technical abbreviations.
4. <|task:neutral_paraphrase|>: Rewrites verbose or conversational transcripts into neutral, concise statements.

Training Details

• Base Architecture: SmolLM2-135M-Instruct
• Method: Weight-Decomposed Low-Rank Adaptation (DoRA) with prompt loss masking on Apple Silicon (mlx-lm). Fused into standalone weights.
• Style Constraints: Banned conversational fluff ("delve", "in summary", "tapestry") and em dashes (—, –).

---

Usage

1. With MLX-LM (Apple Silicon)

from mlx_lm import load, generate

model_id = "vikramlingam/SmolLM2-135M-Post-ASR-Engine"
model, tokenizer = load(model_id)

prompt = """<|im_start|>system
You are a concise, neutral post-ASR formatting engine. Output clean formatting strictly following the task prefix token. Do not include introductory or concluding conversational filler, AI buzzwords, or em dashes.<|im_end|>
<|im_start|>user
<|task:format_email|> hey team quick heads up standup is moved to tomorrow at eleven am let me know if that works<|im_end|>
<|im_start|>assistant
"""

response = generate(model, tokenizer, prompt=prompt, max_tokens=256)
print(response)

2. With Hugging Face Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "vikramlingam/SmolLM2-135M-Post-ASR-Engine"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32)

messages = [
    {"role": "system", "content": "You are a concise, neutral post-ASR formatting engine. Output clean formatting strictly following the task prefix token. Do not include introductory or concluding conversational filler, AI buzzwords, or em dashes."},
    {"role": "user", "content": "<|task:bullet_points|> alex will finish the auth service by wednesday and maria will write load tests on staging"},
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
### 3. With llama.cpp / GGUF (Q4_K_M)

```python
from llama_cpp import Llama

# Download SmolLM2-135M-Post-ASR-Q4_K_M.gguf from this repo
llm = Llama.from_pretrained(
    repo_id="vikramlingam/SmolLM2-135M-Post-ASR-Engine",
    filename="SmolLM2-135M-Post-ASR-Q4_K_M.gguf",
    n_ctx=1024,
    n_gpu_layers=-1,
    verbose=False,
)

prompt = """<|im_start|>system
You are a concise, neutral post-ASR formatting engine. Output clean formatting strictly following the task prefix token. Do not include introductory or concluding conversational filler, AI buzzwords, or em dashes.<|im_end|>
<|im_start|>user
<|task:format_email|> hey mark we need to shift standup to eleven am please check slide four<|im_end|>
<|im_start|>assistant
"""

output = llm(prompt, max_tokens=256, stop=["<|im_end|>"], temperature=0.0)
print(output["choices"][0]["text"].strip())