README.md

---
license: apache-2.0
language:
  - zh
  - en
tags:
  - speech
  - audio
  - full-duplex
  - realtime
  - speech-to-speech
  - vllm
---

# Lychee-FD

Full-duplex realtime speech interaction model and Docker demo.

[![GitHub](https://img.shields.io/badge/GitHub-Lychee--FD-black?logo=github)](https://github.com/HITsz-TMG/Lychee-FD)
[![Paper](https://img.shields.io/badge/Paper-ACL%202026-red?logo=adobeacrobatreader)](https://aclanthology.org/2026.acl-long.419.pdf)
[![License](https://img.shields.io/badge/License-Apache--2.0-green)](https://github.com/HITsz-TMG/Lychee-FD/blob/main/LICENSE)

This repository hosts the Lychee-FD full-duplex model checkpoint. The Docker
runtime and demo code are maintained in the GitHub repository:

```text
https://github.com/HITsz-TMG/Lychee-FD
```

## Required Weights

The Docker demo needs two weight directories:

| Component | Source | Expected local directory |
| --- | --- | --- |
| Lychee-FD full-duplex model | `HIT-TMG/Lychee-FD`, folder `lychee_full_duplex_v1.5/` | `lychee_full_duplex_v1.5/` |
| Token2Wav vocoder | `stepfun-ai/Step-Audio-2-mini`, folder `token2wav/` | `token2wav/` |

The Lychee-FD checkpoint and Token2Wav checkpoint are intentionally separated.
This model repository provides the Lychee-FD full-duplex checkpoint. Please
download the Token2Wav folder from Step-Audio-2-mini separately.

## Download Weights

Create one local model root and put both directories under it:

```text
/path/to/model-root/
  lychee_full_duplex_v1.5/
  token2wav/
```

Download the Lychee-FD checkpoint:

```bash
huggingface-cli download HIT-TMG/Lychee-FD \
  --include "lychee_full_duplex_v1.5/*" \
  --local-dir /path/to/model-root
```

Download Token2Wav from Step-Audio-2-mini:

```bash
huggingface-cli download stepfun-ai/Step-Audio-2-mini \
  --include "token2wav/*" \
  --local-dir /path/to/model-root
```

After downloading, the directory should look like:

```text
/path/to/model-root/
  lychee_full_duplex_v1.5/
  token2wav/
```

## Docker Quick Start

Clone the GitHub code repository:

```bash
git clone https://github.com/HITsz-TMG/Lychee-FD.git
cd Lychee-FD
```

Create a local Docker environment file:

```bash
cp .env.docker.example .env
```

Edit `.env` and replace the model paths:

```dotenv
LYCHEE_FD_IMAGE=ghcr.io/idealistxy/lychee-fd:latest

# Host path on your machine.
HOST_MODEL_ROOT=/path/to/model-root

# Container paths. The host model root is mounted as /models.
CONTAINER_MODEL_ROOT=/models
ALLOWED_MODEL_ROOT=/models

# Main Lychee-FD checkpoint.
STEPAUDIO_MODEL_PATH=/models/lychee_full_duplex_v1.5

# Token2Wav checkpoint downloaded from stepfun-ai/Step-Audio-2-mini.
STEPAUDIO_T2W_MODEL_PATH=/models/token2wav
```

Important path rule:

- `HOST_MODEL_ROOT` is the path on your host machine.
- `STEPAUDIO_MODEL_PATH` and `STEPAUDIO_T2W_MODEL_PATH` are paths inside the
  container.
- By default, Docker mounts `HOST_MODEL_ROOT` to `/models`, so model paths inside
  `.env` should usually start with `/models`.

Start the demo:

```bash
docker compose pull
docker compose up
```

Open the frontend:

```text
http://127.0.0.1:8084
```

For a remote server, replace `127.0.0.1` with the server IP.

## GPU Settings

By default, Token2Wav and the main backend use separate GPUs:

```dotenv
TOKEN2WAV_CUDA_VISIBLE_DEVICES=0
BACKEND_CUDA_VISIBLE_DEVICES=1
```

For a single-GPU machine, set both to `0`:

```dotenv
TOKEN2WAV_CUDA_VISIBLE_DEVICES=0
BACKEND_CUDA_VISIBLE_DEVICES=0
```

If GPU memory is limited, reduce the vLLM maximum context length:

```dotenv
STEPAUDIO_VLLM_MAX_MODEL_LEN=8192
```

## Model Presets

The frontend model list is loaded from `model_presets_dev.json` in the GitHub
repository. If you customize the model list, make sure the preset path uses the
container path:

```json
{
  "name": "lychee_full_duplex_v1.5",
  "model_path": "/models/lychee_full_duplex_v1.5",
  "backend_type": "vllm",
  "mode": "stable"
}
```

After editing presets:

```bash
docker compose restart frontend
```

## Notes

- The Docker image contains the runtime environment and demo code, but it does
  not include model weights.
- The Token2Wav checkpoint is provided by `stepfun-ai/Step-Audio-2-mini`; please
  follow the license and usage terms of the upstream model repository.
- For source code, Docker Compose files, and detailed serving instructions, see
  the GitHub repository: `https://github.com/HITsz-TMG/Lychee-FD`.