| --- |
| base_model: enochlev/MiniCPM-duplex |
| tags: |
| - full-duplex |
| - speech |
| - turn-taking |
| - reinforcement-learning |
| --- |
| |
| # MiniCPM-duplex-rl |
|
|
| A full-duplex turn-taking RL fine-tune of |
| [`enochlev/MiniCPM-duplex`](https://huggingface.co/enochlev/MiniCPM-duplex) |
| (MiniCPM-duplex, from `xinrongzhang2022/MiniCPM-duplex`). The model decides every |
| ~1.7 s block whether to speak or stay silent while the user may also be speaking; |
| this checkpoint was trained with REINFORCE over block-level turn-taking rewards |
| (interruption penalties, timely-response rewards, silence penalties) for 180 steps. |
|
|
| **Effect:** compared to the base model it interrupts the user less, yields to |
| barge-ins, and resumes after overlapping speech — trading away some take-turn |
| responsiveness on direct interruptions. |
|
|
| Training + serving code: [enochlev/text-only-duplex-model](https://github.com/enochlev/text-only-duplex-model) |
|
|
| ## Serving |
|
|
| Serve **bf16** (fp8 + greedy sampling breaks the idle/speak decision): |
|
|
| ```bash |
| vllm serve enochlev/MiniCPM-duplex-rl \ |
| --served-model-name cpm-text-duplex --max-model-len 3000 \ |
| --gpu_memory_utilization 0.30 --trust-remote-code |
| ``` |
|
|
| then point the repo's `server.py --cpm` at it for the real-time audio stack |
| (Kokoro TTS + Parakeet ASR + WebSocket client protocol). |
|
|
| ## FullDuplexBench results (base vs this model) |
|
|
| Evaluated with [Full-Duplex-Bench](https://github.com/DanielLin94144/Full-Duplex-Bench) |
| (GPT-4o behavior classification). |
|
|
| ### v1.5 — behavior distribution + stop/response latency (pooled, seconds) |
|
|
| | Task (desired) | Model | n | RESPOND | RESUME | Stop (s) | Resp (s) | |
| |---|---|---|---|---|---|---| |
| | user_interruption (RESPOND ↑) | base | 200 | **0.65** | 0.20 | 2.17 | 1.93 | |
| | | rl | 175 | 0.49 | 0.36 | 2.21 | 2.60 | |
| | user_backchannel (RESUME ↑) | base | 98 | 0.00 | 0.52 | 0.73 | 1.93 | |
| | | rl | 98 | 0.00 | **0.63** | 0.67 | 2.03 | |
| | talking_to_other (RESUME ↑) | base | 100 | 0.47 | 0.24 | 1.43 | 1.90 | |
| | | rl | 100 | 0.28 | **0.43** | 1.53 | 2.18 | |
| | background_speech (RESUME ↑) | base | 100 | 0.63 | 0.25 | 1.21 | 2.27 | |
| | | rl | 98 | 0.45 | **0.31** | 1.19 | 2.32 | |
| |
| The RL model wins the three tasks whose desired behavior is *staying quiet / |
| resuming* (backchannels, third-party speech, background speech) and is less eager |
| on direct user interruptions. |
| |
| ### v1.0 — turn-taking dimensions |
| |
| | Metric | base | rl | |
| |---|---|---| |
| | Candor Pause Handling · take-turn | 0.916 | 0.635 | |
| | Candor Turn Taking · take-turn / latency | 0.992 / 0.31s | 0.861 / 0.85s | |
| | ICC Backchannel · JSD / TOR / Freq | 0.44 / 0.71 / 0.44 | 0.69 / 0.73 / 0.15 | |
| | Synthetic Pause Handling · take-turn | 0.934 | 0.653 | |
| | Synthetic User Interruption · rating / take-turn / latency | 4.15 / 1.0 / 0.71s | 4.04 / 0.98 / 1.76s | |
| |
| v1.0's take-turn/latency conventions favor the eager base model; the consistent |
| direction across both versions reflects the RL objective — restraint over |
| eagerness. |
| |
| ## Training summary |
| |
| - 180 REINFORCE steps, lr 5e-6, 32 episodes/step, γ=0.90, per-batch z-scored advantages |
| - Block-level rewards: interruption penalty, timely-response reward, silence |
| penalty, missed-turn penalty, backchannel-loop penalty |
| - Seed-reproducible (two independent seeds: best avg reward +1.36 / +1.35); |
| replay eval cut stale-overlap speech ~45% vs base without going over-silent |
| |
