Instructions to use allensung1226/townpass-assistant-models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use allensung1226/townpass-assistant-models with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible embeddings server: llama serve -hf allensung1226/townpass-assistant-models:Q8_0 --embeddings
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible embeddings server: llama serve -hf allensung1226/townpass-assistant-models:Q8_0 --embeddings
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible embeddings server: ./build/bin/llama serve -hf allensung1226/townpass-assistant-models:Q8_0 --embeddings
- Llama new
- LM Studio
- Jan
- Ollama
How to use allensung1226/townpass-assistant-models with Ollama:
ollama run hf.co/allensung1226/townpass-assistant-models:Q8_0
- Unsloth Desktop
- Docker Model Runner
How to use allensung1226/townpass-assistant-models with Docker Model Runner:
docker model run hf.co/allensung1226/townpass-assistant-models:Q8_0
- Lemonade
How to use allensung1226/townpass-assistant-models with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull allensung1226/townpass-assistant-models:Q8_0
Run and chat with the model
lemonade run user.townpass-assistant-models-Q8_0
List all available models
lemonade list
- Atomic Chat
TownPass on-device assistant models
Model files downloaded by the 台北一點就通!微服務入口整合助理 (an on-device assistant for the TownPass / 台北通 app fork, https://github.com/goodmorning1226/TownPass) on first use, after the user agrees. Everything runs on the phone; no recording or text is sent anywhere. Only files we converted or built are here; the other models (Whisper tiny/small, Ministral 3B, Gemma 4 E2B) are downloaded from their authors' own Hugging Face repositories.
The App checks every file's size and SHA-256 against its built-in manifest (assets/assistant/models.json) before
using it.
本 repo 放的是「台北一點就通!微服務入口整合助理」在手機上執行時,使用者同意後才下載的模型檔。 只放我們自己轉檔或建立的檔案;其餘模型從原作者的 Hugging Face 下載。每個檔案的授權如下表。
| Path | Size | SHA-256 | Original | License | What we did |
|---|---|---|---|---|---|
e5/multilingual-e5-small-q8_0.gguf |
132,441,760 | 1d76aaf0627f8385090b4027fcd3827861e7d51f1e27c9f0f479551b6da810d3 |
intfloat/multilingual-e5-small (Microsoft) | MIT, see LICENSES/MIT-multilingual-e5.txt |
Converted to GGUF and quantized to q8_0 for llama.cpp |
decision/townpass-decision-e5-ce-v1-q8_0.gguf |
132,441,120 | ae1745a7a24c1217271f08e53f140df1ea9e3c50f098362045625290d6b6e22f |
Fine-tuned by this project from intfloat/multilingual-e5-small (Microsoft) | MIT, see LICENSES/MIT-multilingual-e5.txt |
Trained as a cross-encoder that scores a request against one App page (decision classifier), on synthetic requests written for this project plus the App's own page names, keywords and examples; evaluation sentences were kept out. Converted to GGUF, quantized to q8_0 |
faq_rag/embeddings.bin |
12,473,856 | 483043bfeddb364bcd43a84866341f99b552e861d10f1c74df04a670028c9a19 |
Embeddings of the Taipei City FAQ (below) by the e5 model above | Open Government Data License, version 1.0 (data) | 8,121 entries × 384 float32, normalized, row-major |
faq_rag/metadata.json |
6,572,186 | b771de1a423dcb13ae504e49d0f522499ed463085810a86aeb9ab31410e0bbd4 |
臺北市市政網站整合平台之常見問答, Taipei City Government | Open Government Data License, version 1.0 | Questions, answers and departments, as published |
faq_rag/model_info.json |
409 | 999d8469d836348696b62882e13fc2c3097937b7ce4c9c3cb24cb8c9bcd049c1 |
— | — | Index description (model, prefixes, dimensions) |
vietnamese/ggml-phowhisper-base-q8_0.bin |
81,768,602 | 9e78100d845978524462f17c2a2f22770e288393719973056033526335621d2a |
vinai/PhoWhisper-base (VinAI) | BSD-3-Clause, see LICENSES/BSD-3-Clause-PhoWhisper.txt |
Converted to whisper.cpp ggml and quantized to q8_0 |
taiwanese/ggml-breeze-asr-26-q8_0.bin |
1,656,129,708 | e878f611bf3ef407cff3c1e2ecf5f60dc92bc6dd09a85b22166731c0d92764a1 |
MediaTek-Research/Breeze-ASR-26 (MediaTek Research) | Apache-2.0, see LICENSES/Apache-2.0.txt |
Converted with whisper.cpp convert-h5-to-ggml.py, quantized to q8_0 with whisper-quantize |
taiwanese/ggml-breeze-asr-26-q4_k.bin |
888,932,908 | 747896198b388e1ab77397cb82136dd3bd5fc1823f7e0c60757166b20ff9abbe |
same as above | Apache-2.0 | Same conversion, quantized to q4_K (used by App builds before 2026-11-13) |
taiwanese/ggml-breeze-asr-26-q4_0.bin |
888,932,908 | 0e08c5a2d3a5ba4bae7f85d02212e887537f78d7e623d8694313e2dd65de22ec |
same as above | Apache-2.0 | Same conversion, quantized to q4_0 (iPhone, and Android phones with 8 GB of memory, from 2026-11-13: as accurate as q8_0 in our evaluation, less memory) |
Conversions change only the file format and numeric precision, not what the models were trained on (the decision classifier above is the one file we trained ourselves). They are not endorsed by the original authors.
Attribution
- Taipei City FAQ data: 本資料依「政府資料開放授權條款-第1版」提供,來源:臺北市政府(臺北市市政網站整合平台之常見問答)。 The FAQ data is provided under the Open Government Data License, version 1.0 (Taiwan); source: Taipei City Government.
- Breeze-ASR-26: MediaTek Research. Please cite the original model card and paper (Lan et al., 2026, arXiv:2603.19259).
- PhoWhisper: VinAI Research.
- multilingual-e5-small: Wang et al., Microsoft (also the base of the decision classifier).
Use
These files are loaded by whisper.cpp whisper-server (speech) and llama.cpp llama-server --embedding (e5) and
llama-server --reranking --pooling rank (decision classifier) inside the App. The App downloads a path here as https://huggingface.co/allensung1226/townpass-assistant-models/resolve/main/<path>
and checks its SHA-256 before use.
- Downloads last month
- 153
8-bit