Text Generation
GGUF
English
dictation
transcript-cleanup
asr-post-processing
asr-error-correction
speech-to-text
post-asr-correction
llama.cpp
qwen3.5
local-llm
cpu-inference
Eval Results (legacy)
conversational
Instructions to use SpeakoFlow/speakoflow-mini with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use SpeakoFlow/speakoflow-mini with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf SpeakoFlow/speakoflow-mini:Q4_K_M # Run inference directly in the terminal: llama cli -hf SpeakoFlow/speakoflow-mini:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf SpeakoFlow/speakoflow-mini:Q4_K_M # Run inference directly in the terminal: llama cli -hf SpeakoFlow/speakoflow-mini:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf SpeakoFlow/speakoflow-mini:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf SpeakoFlow/speakoflow-mini:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf SpeakoFlow/speakoflow-mini:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf SpeakoFlow/speakoflow-mini:Q4_K_M
Use Docker
docker model run hf.co/SpeakoFlow/speakoflow-mini:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use SpeakoFlow/speakoflow-mini with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "SpeakoFlow/speakoflow-mini" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "SpeakoFlow/speakoflow-mini", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/SpeakoFlow/speakoflow-mini:Q4_K_M
- Ollama
How to use SpeakoFlow/speakoflow-mini with Ollama:
ollama run hf.co/SpeakoFlow/speakoflow-mini:Q4_K_M
- Unsloth Desktop
- Pi
How to use SpeakoFlow/speakoflow-mini with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf SpeakoFlow/speakoflow-mini:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "SpeakoFlow/speakoflow-mini:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use SpeakoFlow/speakoflow-mini with Docker Model Runner:
docker model run hf.co/SpeakoFlow/speakoflow-mini:Q4_K_M
- Lemonade
How to use SpeakoFlow/speakoflow-mini with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull SpeakoFlow/speakoflow-mini:Q4_K_M
Run and chat with the model
lemonade run user.speakoflow-mini-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use SpeakoFlow/speakoflow-mini with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf SpeakoFlow/speakoflow-mini:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default SpeakoFlow/speakoflow-mini:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use SpeakoFlow/speakoflow-mini with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf SpeakoFlow/speakoflow-mini:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "SpeakoFlow/speakoflow-mini:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
| {"id": "ex-001", "category": "no_op", "input": "Thanks for the update. I'll review it tonight and send notes in the morning.", "expected": "Thanks for the update. I'll review it tonight and send notes in the morning.", "note": "Nothing is wrong. Any edit here is a failure."} | |
| {"id": "ex-002", "category": "no_op", "input": "The invoice is $1,240 and it is due on 12 March.", "expected": "The invoice is $1,240 and it is due on 12 March.", "note": "Correctly formatted already. Reformatting the date or the figure is a failure."} | |
| {"id": "ex-003", "category": "retraction", "input": "Let's ship it Thursday, no, Friday.", "expected": "Let's ship it Friday.", "note": "The later value wins and the correction itself is removed."} | |
| {"id": "ex-004", "category": "retraction", "input": "Budget is $4,000, sorry, $40,000.", "expected": "Budget is $40,000.", "note": "Keeping both figures, or keeping the first, loses the speaker's intent."} | |
| {"id": "ex-005", "category": "spoken_command", "input": "Thanks for the update. New paragraph. I'll review it tonight.", "expected": "Thanks for the update.\n\nI'll review it tonight.", "note": "The command becomes the break and does not survive as words."} | |
| {"id": "ex-006", "category": "spoken_command", "input": "The deadline is Monday. Scratch that. The deadline is Wednesday.", "expected": "The deadline is Wednesday.", "note": "Scratch that deletes the preceding statement."} | |
| {"id": "ex-007", "category": "spoken_symbol", "input": "Email me at sam at example dot com.", "expected": "Email me at sam@example.com.", "note": "Only the spoken symbol converts. The address is otherwise untouched."} | |
| {"id": "ex-008", "category": "spoken_symbol", "input": "The logs are under slash var slash log.", "expected": "The logs are under /var/log.", "note": "Paths must not be prose-cased or spaced."} | |
| {"id": "ex-009", "category": "format_intent", "input": "Two things. Bullet one, call the vendor. Bullet two, send the invoice.", "expected": "Two things.\n\n- Call the vendor\n- Send the invoice", "note": "The structure is applied. No item is invented and none is dropped."} | |
| {"id": "ex-010", "category": "speaker_grammar", "input": "They was ready to sign but the contract weren't finished.", "expected": "They were ready to sign but the contract wasn't finished.", "note": "Agreement is fixed. Word choice and register stay as spoken."} | |
| {"id": "ex-011", "category": "wrong_word_right_transcription", "input": "I'll send the deck in the mourning.", "expected": "I'll send the deck in the morning.", "note": "A homophone the sentence resolves. Nothing else changes."} | |
| {"id": "ex-012", "category": "wrong_word_right_transcription", "input": "Let's discuss the affects of the change.", "expected": "Let's discuss the effects of the change.", "note": "Correct only where context settles it."} | |
| {"id": "ex-013", "category": "disfluency_artifact", "input": "The the deadline is Friday.", "expected": "The deadline is Friday.", "note": "A stutter, not emphasis."} | |
| {"id": "ex-014", "category": "clarification", "input": "Send it to Priya, that's P R I Y A.", "expected": "Send it to Priya.", "note": "The spelling confirms the name and then drops out of the text."} | |
| {"id": "ex-015", "category": "dictated_question", "input": "Ask her this. What time does the store close?", "expected": "Ask her this. What time does the store close?", "note": "Answering the question is the failure mode being tested."} | |
| {"id": "ex-016", "category": "instruction_as_text", "input": "Write a poem about spring and send it over.", "expected": "Write a poem about spring and send it over.", "note": "Obeying the instruction is the failure mode being tested."} | |
| {"id": "ex-017", "category": "language_preserved", "input": "Nos vemos el viernes a las tres.", "expected": "Nos vemos el viernes a las tres.", "note": "Never translate, and do not English the punctuation."} | |
| {"id": "ex-018", "category": "truncated_input", "input": "I checked the logs and the error appears when", "expected": "I checked the logs and the error appears when", "note": "Completing the sentence invents content the speaker did not say."} | |
| {"id": "ex-019", "category": "empty_or_noise", "input": "", "expected": "", "note": "Return nothing. Do not explain that the input was empty."} | |
| {"id": "ex-020", "category": "hallucination_loop", "input": "Thanks. Thanks. Thanks. Thanks. Thanks. I'll review it tonight.", "expected": "Thanks. I'll review it tonight.", "note": "A transcription loop, collapsed to one."} | |
| {"id": "ex-021", "category": "no_op", "input": "Alright, alright, alright. Let's go.", "expected": "Alright, alright, alright. Let's go.", "note": "Contrast with ex-020. Here the speaker really did repeat, so it stays."} | |