File size: 4,447 Bytes
2e718c9 321e0d7 79894b0 321e0d7 2e718c9 321e0d7 5cd1969 321e0d7 a61fb30 321e0d7 a61fb30 321e0d7 a61fb30 321e0d7 a61fb30 321e0d7 d79d6bd 321e0d7 a61fb30 321e0d7 a61fb30 321e0d7 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 | ---
license: apache-2.0
language:
- zh
- en
tags:
- speech
- audio
- full-duplex
- realtime
- speech-to-speech
- vllm
---
# Lychee-FD
Full-duplex realtime speech interaction model and Docker demo.
[](https://github.com/HITsz-TMG/Lychee-FD)
[](https://hitsz-tmg.github.io/Lychee-FD/)
[](https://arxiv.org/abs/2607.06540)
[](https://github.com/HITsz-TMG/Lychee-FD/blob/main/LICENSE)
This repository hosts the Lychee-FD full-duplex model checkpoint. The Docker
runtime and demo code are maintained in the GitHub repository:
```text
https://github.com/HITsz-TMG/Lychee-FD
```
## Required Weights
The Docker demo needs two weight directories:
| Component | Source | Expected local directory |
| --- | --- | --- |
| Lychee-FD full-duplex model | `HIT-TMG/Lychee-FD`, folder `lychee_full_duplex/` | `lychee_full_duplex/` |
| Token2Wav vocoder | `stepfun-ai/Step-Audio-2-mini`, folder `token2wav/` | `token2wav/` |
The Lychee-FD checkpoint and Token2Wav checkpoint are intentionally separated.
This model repository provides the Lychee-FD full-duplex checkpoint. Please
download the Token2Wav folder from Step-Audio-2-mini separately.
## Download Weights
Create one local model root and put both directories under it:
```text
/path/to/model-root/
lychee_full_duplex/
token2wav/
```
Download the Lychee-FD checkpoint:
```bash
huggingface-cli download HIT-TMG/Lychee-FD \
--include "lychee_full_duplex/*" \
--local-dir /path/to/model-root
```
Download Token2Wav from Step-Audio-2-mini:
```bash
huggingface-cli download stepfun-ai/Step-Audio-2-mini \
--include "token2wav/*" \
--local-dir /path/to/model-root
```
After downloading, the directory should look like:
```text
/path/to/model-root/
lychee_full_duplex/
token2wav/
```
## Docker Quick Start
Clone the GitHub code repository:
```bash
git clone https://github.com/HITsz-TMG/Lychee-FD.git
cd Lychee-FD
```
Create a local Docker environment file:
```bash
cp .env.docker.example .env
```
Edit `.env` and replace the model paths:
```dotenv
LYCHEE_FD_IMAGE=ghcr.io/hitsz-tmg/lychee-fd:latest
# Host path on your machine.
HOST_MODEL_ROOT=/path/to/model-root
# Container paths. The host model root is mounted as /models.
CONTAINER_MODEL_ROOT=/models
ALLOWED_MODEL_ROOT=/models
# Main Lychee-FD checkpoint.
STEPAUDIO_MODEL_PATH=/models/lychee_full_duplex
# Token2Wav checkpoint downloaded from stepfun-ai/Step-Audio-2-mini.
STEPAUDIO_T2W_MODEL_PATH=/models/token2wav
```
Important path rule:
- `HOST_MODEL_ROOT` is the path on your host machine.
- `STEPAUDIO_MODEL_PATH` and `STEPAUDIO_T2W_MODEL_PATH` are paths inside the
container.
- By default, Docker mounts `HOST_MODEL_ROOT` to `/models`, so model paths inside
`.env` should usually start with `/models`.
Start the demo:
```bash
docker compose pull
docker compose up
```
Open the frontend:
```text
http://127.0.0.1:8084
```
For a remote server, replace `127.0.0.1` with the server IP.
## GPU Settings
By default, Token2Wav and the main backend use separate GPUs:
```dotenv
TOKEN2WAV_CUDA_VISIBLE_DEVICES=0
BACKEND_CUDA_VISIBLE_DEVICES=1
```
For a single-GPU machine, set both to `0`:
```dotenv
TOKEN2WAV_CUDA_VISIBLE_DEVICES=0
BACKEND_CUDA_VISIBLE_DEVICES=0
```
If GPU memory is limited, reduce the vLLM maximum context length:
```dotenv
STEPAUDIO_VLLM_MAX_MODEL_LEN=8192
```
## Model Presets
The frontend model list is loaded from `model_presets_dev.json` in the GitHub
repository. If you customize the model list, make sure the preset path uses the
container path:
```json
{
"name": "lychee_full_duplex",
"model_path": "/models/lychee_full_duplex",
"backend_type": "vllm",
"mode": "stable"
}
```
After editing presets:
```bash
docker compose restart frontend
```
## Notes
- The Docker image contains the runtime environment and demo code, but it does
not include model weights.
- The Token2Wav checkpoint is provided by `stepfun-ai/Step-Audio-2-mini`; please
follow the license and usage terms of the upstream model repository.
- For source code, Docker Compose files, and detailed serving instructions, see
the GitHub repository: `https://github.com/HITsz-TMG/Lychee-FD`.
|