Update full duplex bench
Browse files
README.md
CHANGED
|
@@ -912,60 +912,6 @@ Note: Scores marked with ∗ are from our evaluation; others are cited from refe
|
|
| 912 |
</div>
|
| 913 |
</details>
|
| 914 |
|
| 915 |
-
<details>
|
| 916 |
-
<summary>Click to view speech duplex results.</summary>
|
| 917 |
-
|
| 918 |
-
**FullDuplexBench v1.0** benchmarking turn-taking behavior of full-duplex spoken dialogue models
|
| 919 |
-
|
| 920 |
-
<div align="center">
|
| 921 |
-
<table style="margin: 0px auto;">
|
| 922 |
-
<tr>
|
| 923 |
-
<th nowrap="nowrap" align="left"><b>Model</b></th>
|
| 924 |
-
<th nowrap="nowrap"><b>Backchannel<br>Mean TOR(↓)</b></th>
|
| 925 |
-
<th nowrap="nowrap"><b>Candor Pause<br>Handling TOR(↓)</b></th>
|
| 926 |
-
<th nowrap="nowrap"><b>Synthetic Pause<br>Handling TOR(↓)</b></th>
|
| 927 |
-
<th nowrap="nowrap"><b>Turn Taking<br>TOR(↑)</b></th>
|
| 928 |
-
<th nowrap="nowrap"><b>Turn Taking<br>Latency(↓)</b></th>
|
| 929 |
-
<th nowrap="nowrap"><b>User Interruption<br>Score(↑)</b></th>
|
| 930 |
-
<th nowrap="nowrap"><b>User Interruption<br>TOR(↑)</b></th>
|
| 931 |
-
<th nowrap="nowrap"><b>User Interruption<br>Latency(↓)</b></th>
|
| 932 |
-
</tr>
|
| 933 |
-
<tr>
|
| 934 |
-
<td nowrap="nowrap" align="left">Qwen3-Omni-30B-A3B-Instruct</td>
|
| 935 |
-
<td align="center">n/a</td>
|
| 936 |
-
<td align="center">n/a</td>
|
| 937 |
-
<td align="center">n/a</td>
|
| 938 |
-
<td align="center">n/a</td>
|
| 939 |
-
<td align="center">n/a</td>
|
| 940 |
-
<td align="center">n/a</td>
|
| 941 |
-
<td align="center">n/a</td>
|
| 942 |
-
<td align="center">n/a</td>
|
| 943 |
-
</tr>
|
| 944 |
-
<tr>
|
| 945 |
-
<td nowrap="nowrap" align="left">Gemini-Live</td>
|
| 946 |
-
<td align="center"><ins>0.09</ins></td>
|
| 947 |
-
<td align="center"><ins>0.26</ins></td>
|
| 948 |
-
<td align="center"><ins>0.31</ins></td>
|
| 949 |
-
<td align="center"><ins>0.66</ins></td>
|
| 950 |
-
<td align="center"><b>1.30</b></td>
|
| 951 |
-
<td align="center"><b>3.38</b></td>
|
| 952 |
-
<td align="center"><ins>0.89</ins></td>
|
| 953 |
-
<td align="center"><b>1.18</b></td>
|
| 954 |
-
</tr>
|
| 955 |
-
<tr>
|
| 956 |
-
<td nowrap="nowrap" align="left">MiniCPM-o 4.5-Instruct</td>
|
| 957 |
-
<td align="center"><b>0.04</b></td>
|
| 958 |
-
<td align="center"><b>0.00</b></td>
|
| 959 |
-
<td align="center"><b>0.00</b></td>
|
| 960 |
-
<td align="center"><b>1.00</b></td>
|
| 961 |
-
<td align="center"><ins>1.45</ins></td>
|
| 962 |
-
<td align="center"><ins>3.29</ins></td>
|
| 963 |
-
<td align="center"><b>0.97</b></td>
|
| 964 |
-
<td align="center"><ins>1.95</ins></td>
|
| 965 |
-
</tr>
|
| 966 |
-
</table>
|
| 967 |
-
</div>
|
| 968 |
-
</details>
|
| 969 |
|
| 970 |
<details>
|
| 971 |
<summary>Click to view inference efficiency results.</summary>
|
|
|
|
| 912 |
</div>
|
| 913 |
</details>
|
| 914 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 915 |
|
| 916 |
<details>
|
| 917 |
<summary>Click to view inference efficiency results.</summary>
|