| --- |
| language: |
| - en |
| license: apache-2.0 |
| base_model: HuggingFaceTB/SmolLM2-360M-Instruct |
| tags: |
| - asr |
| - speech-formatting |
| - smollm2 |
| - mlx |
| - formatting-engine |
| --- |
| |
| # SmolLM2-360M-Post-ASR-Engine |
|
|
| A compact, local fine-tune of **HuggingFaceTB/SmolLM2-360M-Instruct** designed to act as a lightweight post-ASR (Automatic Speech Recognition) formatting utility. |
|
|
| ## Purpose & Scope |
|
|
| Raw ASR transcripts often lack punctuation, structure, capitalization, and contain disfluencies. This model transforms raw spoken transcripts into clean, structured text using specific task tokens. |
|
|
| ### Supported Task Tokens |
|
|
| 1. `<|task:format_email|>`: Formats raw spoken notes into a structured email with Subject line and body. |
| 2. `<|task:bullet_points|>`: Extracts concise, action-oriented bullet points from meeting or standup dictations. |
| 3. `<|task:clean_punctuate|>`: Restores punctuation, capitalization, numbers, and technical abbreviations. |
| 4. `<|task:neutral_paraphrase|>`: Rewrites verbose or conversational transcripts into neutral, concise statements. |
|
|
| ### Training Details |
|
|
| - **Base Architecture**: SmolLM2-360M-Instruct |
| - **Method**: Weight-Decomposed Low-Rank Adaptation (DoRA) with prompt loss masking on Apple Silicon (`mlx-lm`). Fused into standalone weights. |
| - **Style Constraints**: Banned conversational fluff ("delve", "in summary", "tapestry") and em dashes (`—`, `–`). |
|
|
| --- |
|
|
| ## Usage |
|
|
| ### 1. With MLX-LM (Apple Silicon) |
|
|
| ```python |
| from mlx_lm import load, generate |
| |
| model_id = "vikramlingam/SmolLM2-360M-Post-ASR-Engine" |
| model, tokenizer = load(model_id) |
| |
| prompt = """<|im_start|>system |
| You are a concise, neutral post-ASR formatting engine. Output clean formatting strictly following the task prefix token. Do not include introductory or concluding conversational filler, AI buzzwords, or em dashes.<|im_end|> |
| <|im_start|>user |
| <|task:format_email|> hey team quick heads up standup is moved to tomorrow at eleven am let me know if that works<|im_end|> |
| <|im_start|>assistant |
| """ |
| |
| response = generate(model, tokenizer, prompt=prompt, max_tokens=256) |
| print(response) |
| ``` |
|
|
| ### 2. With Hugging Face Transformers |
|
|
| ```python |
| import torch |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model_id = "vikramlingam/SmolLM2-360M-Post-ASR-Engine" |
| tokenizer = AutoTokenizer.from_pretrained(model_id) |
| model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32) |
| |
| messages = [ |
| {"role": "system", "content": "You are a concise, neutral post-ASR formatting engine. Output clean formatting strictly following the task prefix token. Do not include introductory or concluding conversational filler, AI buzzwords, or em dashes."}, |
| {"role": "user", "content": "<|task:bullet_points|> alex will finish the auth service by wednesday and maria will write load tests on staging"}, |
| ] |
| |
| prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) |
| inputs = tokenizer(prompt, return_tensors="pt") |
| outputs = model.generate(**inputs, max_new_tokens=256, do_sample=False) |
| print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)) |
| ### 3. With llama.cpp / GGUF (Q4_K_M) |
| |
| ```python |
| from llama_cpp import Llama |
| |
| # Download SmolLM2-360M-Post-ASR-Q4_K_M.gguf from this repo |
| llm = Llama.from_pretrained( |
| repo_id="vikramlingam/SmolLM2-360M-Post-ASR-Engine", |
| filename="SmolLM2-360M-Post-ASR-Q4_K_M.gguf", |
| n_ctx=1024, |
| n_gpu_layers=-1, |
| verbose=False, |
| ) |
| |
| prompt = """<|im_start|>system |
| You are a concise, neutral post-ASR formatting engine. Output clean formatting strictly following the task prefix token. Do not include introductory or concluding conversational filler, AI buzzwords, or em dashes.<|im_end|> |
| <|im_start|>user |
| <|task:format_email|> hey mark we need to shift standup to eleven am please check slide four<|im_end|> |
| <|im_start|>assistant |
| """ |
|
|
| output = llm(prompt, max_tokens=256, stop=["<|im_end|>"], temperature=0.0) |
| print(output["choices"][0]["text"].strip()) |
| ``` |
| |
