docs: update repository paths and identify FunAudioChat base
Browse files
README.md
CHANGED
|
@@ -11,15 +11,15 @@ tags:
|
|
| 11 |
- audio-language-model
|
| 12 |
---
|
| 13 |
|
| 14 |
-
# Awaken Full Duplex
|
| 15 |
|
| 16 |
<div align="center">
|
| 17 |
|
| 18 |
**Awaken: Efficient Full-Duplex Adaptation with Capability Retention**
|
| 19 |
|
| 20 |
-
[](https://avera-labs.github.io/AwakenFullDuplex/)
|
| 21 |
-
[](https://github.com/avera-labs/AwakenFullDuplex)
|
| 22 |
-
[](https://huggingface.co/openmartdev/
|
| 23 |
[](https://huggingface.co/FunAudioLLM/Fun-Audio-Chat-8B)
|
| 24 |
|
| 25 |
</div>
|
|
@@ -29,8 +29,10 @@ tags:
|
|
| 29 |
Full-duplex voice models can listen and speak at the same time, so they can acknowledge,
|
| 30 |
interrupt, and hand over turns the way people do. Teaching this usually depends on natural
|
| 31 |
conversational audio rich in backchannels. That audio is scarce, and without careful curation
|
| 32 |
-
it can weaken a model's reasoning and grounding. **Awaken
|
| 33 |
-
|
|
|
|
|
|
|
| 34 |
|
| 35 |
<p align="center">
|
| 36 |
<img width="95%" src="assets/results.png" alt="Awaken evaluation results: backchannels, groundedness, spoken logic puzzles, and spoken question answering">
|
|
@@ -64,7 +66,7 @@ small amount of such data and keeps its base model's scores.
|
|
| 64 |
on 100,132 records with replay of the interaction data.
|
| 65 |
3. **GRPO**: improves backchannel timing from the model's own samples in conversational windows.
|
| 66 |
|
| 67 |
-
See the [project page](https://
|
| 68 |
|
| 69 |
## Model Details
|
| 70 |
|
|
@@ -89,8 +91,8 @@ See the [project page](https://github.com/avera-labs/AwakenFullDuplex/blob/main/
|
|
| 89 |
## Installation
|
| 90 |
|
| 91 |
```bash
|
| 92 |
-
git clone https://github.com/avera-labs/AwakenFullDuplex
|
| 93 |
-
cd AwakenFullDuplex
|
| 94 |
|
| 95 |
# 1. torch + torchaudio for YOUR CUDA driver first (the default PyPI wheel targets the newest
|
| 96 |
# CUDA and will not load on older drivers), e.g. for CUDA 12.8:
|
|
@@ -107,7 +109,7 @@ pip install -e ".[voice]" # + onnxruntime, to make voice prompts from a clip
|
|
| 107 |
|
| 108 |
```bash
|
| 109 |
pip install huggingface-hub
|
| 110 |
-
hf download openmartdev/
|
| 111 |
```
|
| 112 |
|
| 113 |
The speech codec is fetched automatically from
|
|
@@ -184,7 +186,7 @@ InteractionBench results and the Fun-Audio-Chat technical report.
|
|
| 184 |
| **Awaken GRPO** | **75.6** |
|
| 185 |
|
| 186 |
Because our runs and the published results are scored differently, compare across systems with
|
| 187 |
-
care. Full methodology is on the [project page](https://
|
| 188 |
|
| 189 |
## Inference Details
|
| 190 |
|
|
@@ -242,14 +244,14 @@ awaken_chat/
|
|
| 242 |
scripts/ ws_client.py, a paced WebSocket client that drives the server from a wav file
|
| 243 |
tests/ CPU unit tests on tiny random models
|
| 244 |
hf_model_card/ Hugging Face model card (README + figures)
|
| 245 |
-
|
| 246 |
```
|
| 247 |
|
| 248 |
</details>
|
| 249 |
|
| 250 |
## License
|
| 251 |
|
| 252 |
-
The code and model weights are licensed under the [Apache 2.0 License](https://github.com/avera-labs/AwakenFullDuplex/blob/main/LICENSE), the same license
|
| 253 |
as the base model [Fun-Audio-Chat-8B](https://huggingface.co/FunAudioLLM/Fun-Audio-Chat-8B).
|
| 254 |
|
| 255 |
## Acknowledgments
|
|
@@ -257,9 +259,9 @@ as the base model [Fun-Audio-Chat-8B](https://huggingface.co/FunAudioLLM/Fun-Aud
|
|
| 257 |
The model architecture is derived from [Fun-Audio-Chat](https://github.com/FunAudioLLM/Fun-Audio-Chat)
|
| 258 |
and the speech codec from [CosyVoice](https://github.com/FunAudioLLM/CosyVoice) (both Apache-2.0);
|
| 259 |
the text backbone is [Qwen3](https://github.com/QwenLM/Qwen3), and the ear follows
|
| 260 |
-
[Whisper](https://github.com/openai/whisper)'s encoder. See [`NOTICE`](https://github.com/avera-labs/AwakenFullDuplex/blob/main/NOTICE).
|
| 261 |
|
| 262 |
## Contact
|
| 263 |
|
| 264 |
-
- 🐛 Submit an [Issue](https://github.com/avera-labs/AwakenFullDuplex/issues)
|
| 265 |
-
- 💡 Submit a [Pull Request](https://github.com/avera-labs/AwakenFullDuplex/pulls)
|
|
|
|
| 11 |
- audio-language-model
|
| 12 |
---
|
| 13 |
|
| 14 |
+
# Awaken Full Duplex — Based on FunAudioChat
|
| 15 |
|
| 16 |
<div align="center">
|
| 17 |
|
| 18 |
**Awaken: Efficient Full-Duplex Adaptation with Capability Retention**
|
| 19 |
|
| 20 |
+
[](https://avera-labs.github.io/AwakenFullDuplex-FunAudioChat/)
|
| 21 |
+
[](https://github.com/avera-labs/AwakenFullDuplex-FunAudioChat)
|
| 22 |
+
[](https://huggingface.co/openmartdev/AwakenFullDuplex-FunAudioChat)
|
| 23 |
[](https://huggingface.co/FunAudioLLM/Fun-Audio-Chat-8B)
|
| 24 |
|
| 25 |
</div>
|
|
|
|
| 29 |
Full-duplex voice models can listen and speak at the same time, so they can acknowledge,
|
| 30 |
interrupt, and hand over turns the way people do. Teaching this usually depends on natural
|
| 31 |
conversational audio rich in backchannels. That audio is scarce, and without careful curation
|
| 32 |
+
it can weaken a model's reasoning and grounding. **Awaken is based on
|
| 33 |
+
[FunAudioChat](https://huggingface.co/FunAudioLLM/Fun-Audio-Chat-8B)**, adapting the half-duplex
|
| 34 |
+
model for full-duplex conversation using only a small amount of dialogue data while retaining
|
| 35 |
+
most of the base model's benchmark performance.
|
| 36 |
|
| 37 |
<p align="center">
|
| 38 |
<img width="95%" src="assets/results.png" alt="Awaken evaluation results: backchannels, groundedness, spoken logic puzzles, and spoken question answering">
|
|
|
|
| 66 |
on 100,132 records with replay of the interaction data.
|
| 67 |
3. **GRPO**: improves backchannel timing from the model's own samples in conversational windows.
|
| 68 |
|
| 69 |
+
See the [project page](https://avera-labs.github.io/AwakenFullDuplex-FunAudioChat/) for the full data composition and training details.
|
| 70 |
|
| 71 |
## Model Details
|
| 72 |
|
|
|
|
| 91 |
## Installation
|
| 92 |
|
| 93 |
```bash
|
| 94 |
+
git clone https://github.com/avera-labs/AwakenFullDuplex-FunAudioChat
|
| 95 |
+
cd AwakenFullDuplex-FunAudioChat
|
| 96 |
|
| 97 |
# 1. torch + torchaudio for YOUR CUDA driver first (the default PyPI wheel targets the newest
|
| 98 |
# CUDA and will not load on older drivers), e.g. for CUDA 12.8:
|
|
|
|
| 109 |
|
| 110 |
```bash
|
| 111 |
pip install huggingface-hub
|
| 112 |
+
hf download openmartdev/AwakenFullDuplex-FunAudioChat --local-dir ./pretrained_models/awaken-chat
|
| 113 |
```
|
| 114 |
|
| 115 |
The speech codec is fetched automatically from
|
|
|
|
| 186 |
| **Awaken GRPO** | **75.6** |
|
| 187 |
|
| 188 |
Because our runs and the published results are scored differently, compare across systems with
|
| 189 |
+
care. Full methodology is on the [project page](https://avera-labs.github.io/AwakenFullDuplex-FunAudioChat/).
|
| 190 |
|
| 191 |
## Inference Details
|
| 192 |
|
|
|
|
| 244 |
scripts/ ws_client.py, a paced WebSocket client that drives the server from a wav file
|
| 245 |
tests/ CPU unit tests on tiny random models
|
| 246 |
hf_model_card/ Hugging Face model card (README + figures)
|
| 247 |
+
index.html project page
|
| 248 |
```
|
| 249 |
|
| 250 |
</details>
|
| 251 |
|
| 252 |
## License
|
| 253 |
|
| 254 |
+
The code and model weights are licensed under the [Apache 2.0 License](https://github.com/avera-labs/AwakenFullDuplex-FunAudioChat/blob/main/LICENSE), the same license
|
| 255 |
as the base model [Fun-Audio-Chat-8B](https://huggingface.co/FunAudioLLM/Fun-Audio-Chat-8B).
|
| 256 |
|
| 257 |
## Acknowledgments
|
|
|
|
| 259 |
The model architecture is derived from [Fun-Audio-Chat](https://github.com/FunAudioLLM/Fun-Audio-Chat)
|
| 260 |
and the speech codec from [CosyVoice](https://github.com/FunAudioLLM/CosyVoice) (both Apache-2.0);
|
| 261 |
the text backbone is [Qwen3](https://github.com/QwenLM/Qwen3), and the ear follows
|
| 262 |
+
[Whisper](https://github.com/openai/whisper)'s encoder. See [`NOTICE`](https://github.com/avera-labs/AwakenFullDuplex-FunAudioChat/blob/main/NOTICE).
|
| 263 |
|
| 264 |
## Contact
|
| 265 |
|
| 266 |
+
- 🐛 Submit an [Issue](https://github.com/avera-labs/AwakenFullDuplex-FunAudioChat/issues)
|
| 267 |
+
- 💡 Submit a [Pull Request](https://github.com/avera-labs/AwakenFullDuplex-FunAudioChat/pulls)
|