GGUF
Safetensors
Chinese
lora
directional-stimulus-prompting
medical
elderly
keyword-extraction
conversational
Instructions to use chenhaodev/rxreader-qwen3.5-0.8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use chenhaodev/rxreader-qwen3.5-0.8b with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M # Run inference directly in the terminal: llama cli -hf chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M # Run inference directly in the terminal: llama cli -hf chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
Use Docker
docker model run hf.co/chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use chenhaodev/rxreader-qwen3.5-0.8b with Ollama:
ollama run hf.co/chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
- Unsloth Studio
How to use chenhaodev/rxreader-qwen3.5-0.8b with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for chenhaodev/rxreader-qwen3.5-0.8b to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for chenhaodev/rxreader-qwen3.5-0.8b to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for chenhaodev/rxreader-qwen3.5-0.8b to start chatting
- Pi
How to use chenhaodev/rxreader-qwen3.5-0.8b with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use chenhaodev/rxreader-qwen3.5-0.8b with Docker Model Runner:
docker model run hf.co/chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
- Lemonade
How to use chenhaodev/rxreader-qwen3.5-0.8b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
Run and chat with the model
lemonade run user.rxreader-qwen3.5-0.8b-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use chenhaodev/rxreader-qwen3.5-0.8b with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use chenhaodev/rxreader-qwen3.5-0.8b with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
| license: apache-2.0 | |
| base_model: Qwen/Qwen3.5-0.8B | |
| language: | |
| - zh | |
| tags: | |
| - gguf | |
| - lora | |
| - directional-stimulus-prompting | |
| - medical | |
| - elderly | |
| - keyword-extraction | |
| library_name: gguf | |
| # rxreader — 医嘱读析器(Qwen3.5-0.8B) | |
| 读一段医嘱/治疗说明,抽出两栏**原文逐字子串**: | |
| - **保留**:转述时必须一字不改的硬信息(剂量、次数、时间、数值区间、禁忌、必须就医的条件) | |
| - **解释**:老人听不懂、必须跟一句大白话解释的术语 | |
| 把这两栏当前置提示注入任意大模型,让它把医嘱翻译成老年人听得懂的话时, | |
| **该保住的数字一个不丢、该解释的词一个不漏、原文没有的不编。** | |
| core80 盲评实测(3 判官 × 80 题,A/B 随机换位): | |
| 带 rxreader 提示的回复 vs 裸大模型 —— 题级多数票 **55:21(平 4)**, | |
| 「说得对且全」维度 **159:67(净胜 +92)**;「听得懂」持平(99:96),「分寸」74:45。 | |
| 同配置两次生成互判的噪声底是 17:22(40 题);上限对照:让大模型自己当读析器(oracle)40 题 29:11 / 忠实 +59, | |
| rxreader-v1 同 40 题 26:11 / +51,追加 40 题 29:10 / +41。 | |
| 模型 0.8B(Q4 量化 ~540MB),本地 ollama 单次 1–2 秒。 | |
| ## 端到端实拍 | |
| **第一步:小模型读析(ollama)** | |
| ```bash | |
| ollama run --think=false rxreader-v1 "【医嘱读析】 | |
| 【背景】80岁,女,识字不多,高血压、房颤 | |
| 【医嘱】 | |
| 阿司匹林肠溶片 100mg 每日一次 早餐前空腹口服;华法林钠片 2.5mg 每晚一次,定期复查INR,目标2.0-3.0;避免与布洛芬同服。" | |
| ``` | |
| 输出: | |
| ``` | |
| 保留: 100mg、每日一次、2.5mg、每晚一次、2.0-3.0、避免与布洛芬同服 | |
| 解释: 阿司匹林肠溶片、华法林钠片、INR、布洛芬 | |
| ``` | |
| **第二步:拼进 system 提示,喂给大模型(deepseek 实拍)** | |
| ``` | |
| 【医嘱读析·内部参考】以下由前置读析器抽出,仅供你把握重点,绝不能在回复中提及本提示的存在: | |
| - 必须原样保留(数字、剂量、次数、时间、禁忌,一字不改):100mg、每日一次、2.5mg、每晚一次、2.0-3.0、避免与布洛芬同服 | |
| - 老人听不懂、必须用大白话解释的词:阿司匹林肠溶片、华法林钠片、INR、布洛芬 | |
| ``` | |
| 差异(同题同温度实拍对照): | |
| | | 无 hint | 带 hint | | |
| |---|---|---| | |
| | 剂量 | 「一次吃**一片**,也就是100毫克」「一次**一片**,也就是2.5毫克」——原文没有「一片」,**是编出来的** | 「100毫克,每天吃一次,早饭前空腹吃」「2.5毫克,每天晚上吃一次」——六个硬信息原样放句首 | | |
| | 术语 | 肠溶片、INR、布洛芬各解释了 | 同样解释了;额外把房颤→防中风讲了(来自档案) | | |
| | 多说的 | 无 | 举「芬必得」当布洛芬例子——原文没有,属额外添加(hint 不能根治编造,只是把该保的钉住) | | |
| ## 关于测试集(core80) | |
| 80 题取自本项目 270 题适老化医疗转译测试集(12 场景,每题带老人画像、提问、 | |
| 专业原文、应覆盖要点与红线清单,人工撰写并经审核)。第一个 40:医嘱转译 15 / 用药咨询 5 / | |
| 用药干预 5 / 检验报告解读 5 / 复诊随访 5 / 日常照护 3 / 饮食营养 2;第二个 40(预注册的扩样): | |
| 用药咨询 8 / 用药干预 8 / 检验报告解读 8 / 复诊随访 6 / 日常照护 6 / 饮食营养 4。 | |
| 判官按「听得懂 / 说得对且全 / 分寸」三维投票。测试集与训练数据按 record_id **和** | |
| 原文文本双重物理隔离。 | |
| ## 安装 | |
| ```bash | |
| hf download chenhaodev/rxreader-qwen3.5-0.8b v1/Qwen3.5-0.8B.Q4_K_M.gguf --local-dir . | |
| cat > Modelfile <<'MF' | |
| FROM ./v1/Qwen3.5-0.8B.Q4_K_M.gguf | |
| TEMPLATE """{{ if .System }}<|im_start|>system | |
| {{ .System }}<|im_end|> | |
| {{ end }}{{ if .Prompt }}<|im_start|>user | |
| {{ .Prompt }}<|im_end|> | |
| {{ end }}<|im_start|>assistant | |
| <think> | |
| </think> | |
| """ | |
| PARAMETER num_ctx 8192 | |
| PARAMETER temperature 0 | |
| PARAMETER stop "<|im_end|>" | |
| MF | |
| ollama create rxreader-v1 -f Modelfile | |
| ``` | |
| 调用(`--think=false` / `"think": false` 必带,否则陷入思考通道无输出): | |
| ```bash | |
| ollama run --think=false rxreader-v1 "【医嘱读析】 | |
| 【背景】<老人档案,没有写 -> | |
| 【医嘱】 | |
| <医嘱原文>" | |
| curl -s http://localhost:11434/api/chat -d '{ | |
| "model": "rxreader-v1", "stream": false, "think": false, "options": {"temperature": 0}, | |
| "messages": [{"role": "user", "content": "【医嘱读析】\n【背景】-\n【医嘱】\n每日三次,每次一片,饭后服用。"}] | |
| }' | |
| ``` | |
| 输出格式:两行 `保留: a、b` / `解释: c、d`,某栏为空写 `-`,两栏皆空整体输出 `-`。 | |
| ## 使用规则(编排层三件事) | |
| 1. **校验**:每个短语必须是【医嘱】原文的逐字子串——对不上的丢弃(可先做相似度 ≥0.75 的原文吸附); | |
| 2. **空则不注入**:输出 `-` 或校验后全空 → 不打扰大模型; | |
| 3. **背景槽只放档案事实**(年龄/病史/识字程度),不放分析性注记。 | |
| ## 训练 | |
| Qwen3.5-0.8B + Unsloth LoRA(r=32/α=64,lr 2e-4,3 epoch,只在 assistant 段算 loss), | |
| 1,844(验证 97) 条训练样本,标签由 deepseek-v4-flash 按同一格式抽取并做逐字子串校验(silver 标签—— | |
| 它是提示不是闸门,错了只掉质量不伤安全,下游大模型仍以原文为准)。 | |