Instructions to use mlboydaisuke/Dia2-1B-LiteRT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT
How to use mlboydaisuke/Dia2-1B-LiteRT with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Moshi
How to use mlboydaisuke/Dia2-1B-LiteRT with Moshi:
# pip install moshi # Run the interactive web server python -m moshi.server --hf-repo "mlboydaisuke/Dia2-1B-LiteRT" # Then open https://localhost:8998 in your browser
# pip install moshi import torch from moshi.models import loaders # Load checkpoint info from HuggingFace checkpoint = loaders.CheckpointInfo.from_hf_repo("mlboydaisuke/Dia2-1B-LiteRT") # Load the Mimi audio codec mimi = checkpoint.get_mimi(device="cuda") mimi.set_num_codebooks(8) # Encode audio (24kHz, mono) wav = torch.randn(1, 1, 24000 * 10) # [batch, channels, samples] with torch.no_grad(): codes = mimi.encode(wav.cuda()) decoded = mimi.decode(codes) - Dia2
How to use mlboydaisuke/Dia2-1B-LiteRT with Dia2:
from dia2 import Dia2, GenerationConfig, SamplingConfig dia = Dia2.from_repo("mlboydaisuke/Dia2-1B-LiteRT", device="cuda", dtype="bfloat16") config = GenerationConfig( cfg_scale=2.0, audio=SamplingConfig(temperature=0.8, top_k=50), use_cuda_graph=True, ) result = dia.generate("[S1] Hello Dia2!", config=config, output_wav="hello.wav", verbose=True) - Notebooks
- Google Colab
- Kaggle
| {"frames": 71, "aligned": [[799, 915, 1024, 1139, 1197, 1378, 330, 1232, 1237, 510, 1252, 1605, 1860, 1882, 1406, 179, 438, 1532, 378, 1052, 733, 1473, 1473, 126, 2042, 448, 84, 1850, 1850, 1850, 1850, 1116, 330, 46, 392, 1941, 125, 1781, 1211, 607, 607, 59, 1251, 1513, 1513, 144, 1945, 252, 468, 808, 463, 561, 1762, 417, 245, 1165, 774, 1702, 1677, 502, 978, 1743, 1544, 1544, 1544, 1934, 1934, 1934, 1081, 1023, 1560], [302, 637, 1210, 1920, 545, 1686, 1804, 152, 920, 907, 454, 1485, 1596, 647, 1280, 2033, 96, 1227, 661, 1645, 490, 412, 151, 1052, 1515, 243, 243, 243, 243, 243, 243, 1935, 1106, 1931, 116, 546, 908, 1269, 1025, 1417, 1306, 1366, 446, 989, 1222, 173, 1974, 261, 1229, 1881, 760, 156, 339, 700, 1259, 145, 1964, 1255, 425, 477, 1056, 1056, 1056, 1056, 1056, 1056, 1056, 1050, 750, 614, 373], [1690, 725, 24, 784, 1383, 1262, 215, 31, 1217, 924, 253, 738, 1000, 1183, 441, 140, 2001, 1633, 820, 685, 527, 107, 77, 328, 1626, 783, 783, 783, 783, 783, 783, 308, 1449, 782, 1453, 1758, 1929, 301, 1706, 293, 1823, 1392, 1600, 528, 1377, 1365, 199, 838, 480, 1538, 970, 1202, 1856, 660, 295, 1577, 976, 579, 1450, 480, 1697, 1029, 1029, 1559, 1559, 1559, 1029, 1697, 1501, 957, 754], [619, 1830, 1414, 264, 846, 910, 63, 1260, 1340, 114, 890, 723, 1381, 989, 148, 1763, 1003, 293, 1232, 810, 1412, 1738, 1238, 966, 142, 546, 546, 142, 142, 142, 142, 330, 576, 536, 1829, 1511, 2017, 2027, 1447, 830, 674, 921, 1489, 245, 31, 786, 1964, 477, 1135, 505, 901, 721, 191, 1296, 855, 1392, 195, 176, 30, 114, 290, 142, 142, 290, 290, 290, 142, 390, 1744, 1078, 1171], [987, 188, 1031, 987, 1444, 1528, 1146, 1881, 1016, 1761, 854, 497, 1039, 1763, 1666, 1438, 720, 643, 935, 313, 403, 1315, 52, 1019, 306, 267, 267, 267, 481, 481, 481, 1895, 1253, 958, 859, 1240, 1662, 1209, 849, 909, 781, 831, 599, 204, 1287, 872, 548, 1247, 893, 1490, 720, 1900, 1172, 1114, 892, 2005, 1581, 1579, 51, 267, 1736, 267, 267, 481, 1736, 1736, 1736, 340, 2000, 510, 1278], [992, 1610, 924, 1774, 1542, 1691, 489, 224, 724, 2046, 464, 974, 172, 1399, 1173, 1132, 674, 1262, 422, 481, 1209, 1118, 1974, 1851, 11, 555, 1030, 1572, 1572, 555, 1030, 732, 175, 93, 1987, 788, 1995, 175, 768, 1354, 74, 1512, 1016, 1926, 1288, 1774, 1739, 2033, 1919, 547, 1510, 87, 1903, 61, 1697, 182, 1324, 1234, 1929, 11, 347, 1030, 1443, 347, 347, 347, 1443, 478, 56, 80, 1582], [2047, 556, 1243, 553, 1685, 330, 1839, 1161, 1243, 1666, 1412, 967, 422, 1254, 1140, 2039, 119, 1105, 494, 1596, 665, 724, 119, 752, 976, 666, 825, 1978, 666, 666, 666, 259, 883, 1365, 1763, 648, 532, 537, 1415, 291, 266, 1298, 812, 1660, 453, 473, 121, 1708, 1110, 1062, 1432, 209, 1955, 853, 577, 1288, 914, 244, 610, 943, 1978, 1978, 825, 976, 976, 976, 1978, 825, 442, 1229, 1135], [915, 844, 297, 551, 50, 297, 1023, 508, 1939, 299, 372, 1322, 50, 1498, 1339, 1719, 1115, 1245, 1371, 1623, 390, 639, 56, 1949, 2008, 2008, 2008, 2008, 1744, 1648, 1744, 809, 1880, 840, 435, 716, 132, 1098, 1071, 138, 529, 155, 1630, 1214, 928, 888, 633, 770, 1293, 297, 536, 392, 219, 659, 357, 749, 1849, 672, 227, 1852, 2008, 1744, 1744, 2008, 1648, 2008, 1744, 2008, 1329, 1182, 317], [654, 348, 130, 219, 1986, 1438, 1399, 367, 884, 922, 1090, 293, 916, 23, 1196, 153, 435, 435, 793, 1853, 1214, 1685, 369, 1818, 836, 1210, 1210, 739, 739, 739, 739, 1709, 1799, 892, 1763, 1684, 1789, 779, 1388, 254, 953, 1971, 785, 122, 551, 1741, 356, 643, 1409, 1820, 126, 421, 1686, 1765, 1032, 250, 796, 1020, 2016, 469, 1437, 739, 1437, 1437, 739, 739, 1210, 1210, 1409, 753, 1660], [462, 1337, 1285, 61, 952, 1640, 1654, 1481, 1833, 637, 385, 835, 1775, 522, 1511, 2012, 1477, 158, 1656, 833, 1304, 492, 943, 1834, 605, 605, 1992, 605, 605, 1370, 1370, 1723, 949, 1036, 789, 799, 1840, 1321, 1002, 281, 1822, 493, 1109, 522, 879, 867, 1968, 685, 1314, 443, 1705, 1630, 927, 1160, 670, 1918, 106, 1169, 689, 70, 1542, 374, 1370, 1992, 91, 91, 1370, 1187, 1079, 1334, 1415], [635, 1669, 1630, 1669, 731, 1407, 1731, 799, 1588, 1709, 1027, 1430, 1276, 437, 812, 1506, 1725, 1800, 1851, 572, 1977, 1166, 1556, 1750, 1165, 1165, 1165, 415, 1048, 1165, 1165, 1452, 517, 1191, 1264, 1041, 1036, 50, 304, 93, 1726, 1869, 1704, 1304, 111, 1162, 516, 1384, 1277, 1190, 217, 2031, 447, 812, 881, 1654, 579, 29, 1157, 1791, 675, 662, 662, 662, 1048, 662, 662, 1048, 1645, 1091, 1839], [1469, 978, 364, 45, 949, 862, 649, 1977, 931, 1607, 1566, 265, 1890, 368, 1930, 1271, 1395, 1643, 1065, 936, 1040, 872, 178, 95, 436, 439, 436, 1190, 1383, 1383, 1383, 1112, 1079, 1027, 334, 5, 1222, 974, 1923, 845, 1154, 212, 1641, 357, 642, 1933, 240, 120, 1440, 434, 453, 134, 2039, 487, 1150, 893, 1132, 894, 1711, 414, 1190, 1383, 1383, 1190, 439, 1383, 1190, 1383, 1731, 1046, 1907], [335, 2040, 409, 1805, 1241, 512, 1715, 1951, 1502, 6, 907, 790, 1368, 142, 254, 1064, 913, 1591, 1221, 656, 1005, 1764, 1467, 721, 1101, 1101, 1101, 194, 1908, 1908, 1908, 135, 1065, 1111, 783, 1390, 576, 1646, 347, 1201, 669, 234, 202, 283, 1668, 516, 108, 175, 109, 877, 1612, 632, 1293, 329, 1164, 361, 1269, 880, 1565, 2009, 812, 1101, 1101, 1101, 1912, 1343, 1908, 1912, 484, 171, 789], [311, 1469, 1494, 741, 1809, 636, 1027, 806, 1494, 301, 576, 1065, 276, 1981, 602, 1824, 1696, 284, 1630, 1191, 1763, 768, 1317, 1721, 1714, 1430, 650, 113, 113, 1714, 650, 827, 503, 787, 1734, 1218, 1954, 174, 395, 737, 452, 1755, 239, 575, 1363, 1275, 1622, 1183, 982, 1706, 845, 1904, 1259, 625, 1702, 2005, 1594, 416, 1009, 1094, 650, 1204, 650, 650, 1146, 1430, 1714, 113, 1832, 22, 1356], [1809, 1348, 435, 1333, 1951, 1179, 374, 619, 1032, 2039, 824, 1476, 431, 1074, 546, 492, 1091, 836, 444, 1866, 1776, 1705, 792, 1164, 1821, 166, 853, 220, 220, 166, 166, 1177, 267, 1069, 1427, 191, 1168, 1922, 1301, 1337, 156, 402, 157, 676, 1299, 949, 1464, 645, 1343, 683, 1435, 1343, 1326, 2016, 1855, 58, 586, 917, 250, 644, 166, 166, 166, 166, 166, 166, 802, 1653, 1899, 424, 34], [719, 1229, 1059, 1229, 66, 637, 1576, 1165, 261, 1086, 461, 952, 970, 951, 1411, 1959, 1522, 1095, 994, 1042, 228, 722, 1612, 666, 610, 610, 610, 1684, 851, 851, 851, 201, 918, 1306, 267, 1449, 1876, 53, 418, 385, 1374, 892, 1709, 1898, 315, 1511, 1247, 1941, 18, 1999, 1225, 1078, 411, 1500, 1121, 325, 977, 1180, 1587, 1279, 851, 851, 851, 1684, 851, 241, 1684, 851, 123, 914, 1153], [275, 1960, 357, 436, 1877, 648, 1781, 1113, 839, 1063, 684, 746, 1757, 1493, 902, 805, 882, 390, 801, 1949, 1112, 746, 644, 1669, 1531, 32, 1978, 32, 32, 1978, 32, 1840, 1692, 545, 1919, 951, 1280, 309, 853, 1610, 572, 318, 1385, 141, 1289, 625, 671, 1030, 976, 861, 1631, 1041, 1063, 1219, 899, 94, 411, 909, 1728, 532, 32, 32, 1978, 32, 32, 32, 32, 111, 812, 798, 619], [576, 718, 1624, 1236, 1687, 2012, 853, 642, 1143, 486, 741, 761, 864, 1567, 917, 58, 2005, 1309, 1605, 1494, 1265, 1081, 1171, 306, 554, 646, 1642, 1829, 251, 1419, 1419, 1846, 1096, 1146, 1210, 907, 517, 1220, 100, 1124, 196, 150, 1238, 1512, 955, 1676, 1139, 308, 1050, 1619, 1804, 1302, 468, 279, 1676, 1908, 1359, 623, 259, 1829, 1642, 1829, 251, 1829, 251, 1419, 1642, 1419, 1331, 1110, 391], [1172, 501, 562, 1034, 1659, 967, 1470, 1284, 637, 766, 1202, 783, 810, 961, 1364, 1786, 27, 1189, 1066, 254, 1821, 270, 565, 300, 77, 1736, 1736, 1490, 618, 1490, 1736, 1531, 1581, 731, 255, 1691, 1563, 928, 517, 13, 353, 1322, 793, 1596, 809, 1922, 1359, 1667, 24, 827, 1935, 1822, 2039, 616, 1307, 2041, 1536, 423, 1997, 983, 1736, 1490, 290, 290, 618, 1490, 1736, 618, 107, 240, 1839], [1219, 707, 1820, 2024, 152, 1951, 749, 1094, 858, 1333, 495, 766, 458, 1645, 1187, 636, 69, 1521, 306, 1355, 962, 1669, 321, 1460, 1609, 1428, 1406, 1402, 1853, 1428, 1428, 161, 1908, 1423, 768, 292, 383, 687, 859, 1391, 882, 1171, 1861, 1536, 1703, 1790, 228, 1927, 1330, 1254, 1398, 1594, 1458, 1876, 1211, 434, 372, 997, 685, 1626, 1406, 1428, 1406, 1406, 1406, 1402, 1402, 646, 2026, 1873, 323], [1402, 1121, 1666, 1090, 190, 2009, 321, 931, 192, 145, 672, 1701, 274, 740, 1174, 244, 1232, 831, 557, 1801, 1172, 1366, 1114, 902, 1255, 692, 1908, 1255, 692, 1908, 1908, 1115, 139, 922, 1016, 301, 1281, 1122, 1263, 1738, 429, 86, 153, 1396, 431, 1715, 1405, 61, 1504, 1131, 483, 1122, 1957, 1217, 1328, 153, 402, 1946, 568, 1908, 851, 692, 1908, 423, 692, 692, 1908, 1908, 689, 1826, 530], [757, 408, 1988, 1660, 1233, 1513, 538, 877, 714, 386, 1552, 1159, 1680, 406, 1819, 1640, 1121, 88, 1839, 1108, 559, 711, 71, 1672, 1897, 1067, 1626, 1897, 1897, 1626, 1626, 1903, 1777, 32, 1721, 368, 776, 117, 1058, 1908, 998, 396, 1966, 57, 542, 625, 507, 66, 459, 936, 1262, 177, 1338, 1859, 1435, 386, 1021, 657, 1250, 72, 1067, 1953, 1953, 1953, 1626, 1953, 1626, 1897, 357, 476, 712], [1110, 1822, 1363, 1824, 1045, 94, 1577, 933, 1423, 1769, 1806, 1837, 105, 618, 1930, 601, 1117, 1373, 106, 569, 561, 228, 289, 651, 1497, 1353, 1497, 1353, 1332, 1353, 1353, 1647, 514, 1574, 172, 1574, 1890, 1563, 1028, 1799, 1589, 1627, 1031, 64, 374, 796, 749, 47, 1082, 1016, 1914, 921, 13, 1837, 1237, 1209, 1874, 1549, 1795, 634, 1332, 377, 1332, 1497, 377, 1497, 1353, 1453, 1438, 1491, 1589], [1498, 1538, 2045, 609, 1258, 487, 1695, 616, 1755, 803, 326, 1226, 898, 217, 1514, 1080, 73, 383, 844, 112, 1529, 99, 1210, 1067, 1774, 1774, 1586, 717, 1586, 1586, 1586, 516, 227, 1783, 1329, 1175, 1896, 1256, 1266, 1269, 1699, 1371, 1086, 1903, 484, 642, 963, 1384, 839, 1619, 775, 1780, 981, 883, 1694, 969, 392, 1282, 592, 607, 717, 1774, 717, 717, 1586, 1774, 717, 1774, 1545, 1908, 1200], [871, 897, 1294, 1434, 194, 592, 1144, 886, 354, 1192, 755, 178, 946, 1823, 78, 1702, 1417, 473, 1443, 523, 1816, 809, 1327, 781, 172, 724, 724, 204, 204, 1423, 204, 1757, 1234, 1643, 263, 322, 1932, 350, 858, 703, 1844, 1409, 27, 1171, 1413, 831, 792, 569, 1439, 1806, 2019, 672, 1033, 1228, 1167, 219, 462, 1488, 1607, 1673, 724, 724, 724, 1423, 204, 1423, 1423, 1033, 1418, 850, 504], [1301, 1437, 430, 1189, 967, 453, 964, 1592, 449, 1320, 14, 1795, 2020, 567, 1865, 1184, 846, 803, 251, 56, 448, 945, 93, 1810, 1591, 973, 591, 591, 847, 847, 847, 792, 1422, 56, 1425, 342, 1129, 1818, 1025, 1387, 1818, 35, 54, 1907, 37, 293, 1735, 869, 1470, 1211, 1894, 1446, 1786, 1116, 1766, 1664, 607, 1656, 486, 777, 973, 847, 973, 973, 591, 973, 591, 847, 1845, 1862, 790], [1048, 1126, 409, 1225, 404, 107, 678, 634, 1276, 1772, 1072, 1711, 1388, 996, 1605, 1537, 1052, 1650, 76, 1694, 1899, 1650, 1032, 989, 260, 1538, 1538, 569, 569, 569, 569, 871, 1239, 2003, 1032, 543, 345, 1588, 765, 1074, 1144, 784, 1847, 536, 378, 848, 1547, 1246, 928, 1842, 202, 1360, 415, 619, 1093, 189, 252, 858, 538, 883, 569, 1538, 899, 569, 1484, 1893, 569, 1538, 1546, 1316, 318], [1323, 1325, 1444, 1385, 68, 526, 1041, 918, 585, 1314, 754, 206, 1208, 1101, 460, 966, 190, 1295, 604, 1747, 247, 655, 1105, 797, 1388, 951, 483, 1124, 951, 1388, 951, 1181, 1095, 1071, 1130, 1724, 528, 803, 1752, 1127, 703, 859, 254, 313, 46, 1523, 761, 379, 1582, 80, 903, 1792, 1879, 1428, 1514, 1711, 1626, 345, 1307, 1309, 1388, 1388, 951, 1124, 1124, 1124, 483, 951, 1115, 278, 1567], [732, 1654, 869, 1185, 561, 823, 774, 602, 817, 1454, 607, 1204, 1088, 984, 551, 1831, 571, 732, 65, 302, 851, 1679, 2006, 206, 1252, 1995, 35, 1995, 35, 1995, 1995, 2006, 1243, 1287, 147, 1413, 1014, 561, 1867, 379, 540, 1018, 1571, 232, 1621, 48, 78, 378, 1394, 261, 357, 2008, 1615, 1095, 1432, 668, 1032, 854, 1207, 1995, 35, 1995, 35, 422, 66, 422, 422, 35, 1436, 1583, 251], [55, 899, 616, 523, 131, 553, 3, 260, 604, 989, 75, 1316, 1696, 1384, 22, 871, 24, 1126, 17, 309, 1548, 1216, 1212, 496, 1688, 14, 1688, 14, 1440, 97, 1688, 1620, 1448, 597, 1610, 1726, 1809, 1818, 990, 1670, 1697, 339, 1007, 1779, 1581, 216, 754, 1047, 1836, 621, 339, 1955, 1820, 488, 516, 6, 1195, 190, 449, 1658, 1464, 97, 97, 14, 14, 1440, 1440, 1030, 292, 1435, 1001], [345, 954, 1339, 1734, 883, 757, 1905, 1514, 753, 1838, 956, 205, 616, 1964, 420, 1825, 1288, 1572, 1081, 1319, 503, 405, 459, 1952, 1911, 427, 332, 1047, 427, 332, 427, 666, 1984, 300, 899, 634, 1016, 336, 49, 1978, 1001, 667, 585, 415, 1734, 43, 17, 394, 1197, 1548, 399, 17, 1706, 952, 330, 169, 245, 864, 1519, 814, 271, 1140, 1047, 1047, 332, 1140, 427, 1783, 999, 1931, 1088], [1112, 208, 1186, 1325, 1261, 139, 203, 1660, 583, 1048, 911, 1406, 1280, 1750, 824, 973, 1235, 1444, 536, 1116, 1318, 1074, 244, 766, 1718, 1752, 1562, 1833, 1752, 1752, 1752, 231, 242, 1571, 1898, 798, 1242, 1529, 1880, 187, 198, 331, 1709, 1135, 474, 191, 2022, 1679, 1164, 376, 568, 360, 1577, 1035, 1998, 1112, 486, 1651, 1664, 1478, 1780, 1833, 1752, 1562, 1833, 1956, 1956, 1780, 1065, 246, 480]], "new_word_steps": [4, 7, 10, 12, 14, 16, 18, 20, 32, 36, 38, 40, 42, 44, 49, 56], "entries": [{"tokens": [49152, 19103], "padding": 2}, {"tokens": [11527, 28], "padding": 2}, {"tokens": [8232], "padding": 1}, {"tokens": [271], "padding": 1}, {"tokens": [81], "padding": 1}, {"tokens": [38478], "padding": 1}, {"tokens": [39600], "padding": 1}, {"tokens": [10029, 30], "padding": 12}, {"tokens": [49153, 1350], "padding": 3}, {"tokens": [8232], "padding": 1}, {"tokens": [537], "padding": 1}, {"tokens": [271], "padding": 1}, {"tokens": [1195], "padding": 1}, {"tokens": [12247], "padding": 4}, {"tokens": [7567, 5251], "padding": 6}, {"tokens": [100, 3198, 30], "padding": 13}]} |