RVQ Position Aware Speculative Decoding for On Device Text to Speech
Paper • 2609.37007 • Published • 2
Social Post Explorers
community
AI & ML interests
None defined yet.
Recent Activity
View all activity
EduardoPacheco
authored a
paper 7 days ago
Post
119
𝗬𝘂𝗘𝟮 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻 𝘀𝗮𝗺𝗽𝗹𝗲𝘀
Some m-a-p/YuE2-3B samples from my experiments.
All songs are: JA or EN/JA
5️⃣ 𝐎𝐑𝐈𝐆𝐈𝐍𝐀𝐋 (synth-pop)
This song started as a melody SheetSage2 cover of M83 - Midnight City.
Extended for my lyrics, and chords added.
𝘧𝘶𝘭𝘭 𝘴𝘰𝘯𝘨: https://youtu.be/9RX3OLFqV5M
1️⃣ 𝑪𝑶𝑽𝑬𝑹 : BTS - FAKE LOVE (ja version)
𝙛𝙪𝙡𝙡 𝙨𝙤𝙣𝙜: https://youtu.be/fBVmTeSScNg
cleanest sheetsage2-derived song I've gotten.
something that's quite rare about this one is that it's mixed EN and JA. (this has not worked for me in other songs)
4️⃣ 𝐎𝐑𝐈𝐆𝐈𝐍𝐀𝐋 (alt Rock)
𝘧𝘶𝘭𝘭 𝘴𝘰𝘯𝘨: https://youtu.be/D-HXWWCdqNs
Generated from style + lyrics (no user-defined ABC).
Good result, but it is afflicted with the whizz, buzzing artifacting/degeneration on the singing voice, which I get not infrequently.
the singer's pitch and timbre change in a few spots too.
2️⃣ 𝐎𝐑𝐈𝐆𝐈𝐍𝐀𝐋 (synthwave J-Pop)
𝘧𝘶𝘭𝘭 𝘴𝘰𝘯𝘨: https://youtu.be/PYGtp0VLjS8
This one was made without ABC, just letting YuE2 compose a plan before generation.
a few dropped words, odd emphases in certain places, but mostly coherent
3️⃣ 𝑪𝑶𝑽𝑬𝑹 (*attempted*) German
進撃の巨人, vogel im kamfig YouSeeBigGirl:TT
(Reference: https://youtu.be/2o5H8wIR8Go)
I didn't get a decent generation for the song.
but, there was *one* this one section of the song 👏🏻
This happens; you'll have a killer track and *suddenly* some dude starts whispering pidgin language in a weird hermaphroditic AI voice for 2-3 sec, and then the song continues fine, no more cursed artifacts 😕
or, gen is a total trainwreck, but suddenly becomes coherent and makes something cool, like this thing 😌
m-a-p/YuE2-3B
Samples from other models:
- MiniMaxAI/MiniMax-Music3 : https://huggingface.co/posts/evdcush/304645800764284
- ACE-Step/acestep-v15-xl-sft : https://huggingface.co/posts/evdcush/646521189747912
Some m-a-p/YuE2-3B samples from my experiments.
All songs are: JA or EN/JA
5️⃣ 𝐎𝐑𝐈𝐆𝐈𝐍𝐀𝐋 (synth-pop)
This song started as a melody SheetSage2 cover of M83 - Midnight City.
Extended for my lyrics, and chords added.
𝘧𝘶𝘭𝘭 𝘴𝘰𝘯𝘨: https://youtu.be/9RX3OLFqV5M
1️⃣ 𝑪𝑶𝑽𝑬𝑹 : BTS - FAKE LOVE (ja version)
𝙛𝙪𝙡𝙡 𝙨𝙤𝙣𝙜: https://youtu.be/fBVmTeSScNg
cleanest sheetsage2-derived song I've gotten.
something that's quite rare about this one is that it's mixed EN and JA. (this has not worked for me in other songs)
4️⃣ 𝐎𝐑𝐈𝐆𝐈𝐍𝐀𝐋 (alt Rock)
𝘧𝘶𝘭𝘭 𝘴𝘰𝘯𝘨: https://youtu.be/D-HXWWCdqNs
Generated from style + lyrics (no user-defined ABC).
Good result, but it is afflicted with the whizz, buzzing artifacting/degeneration on the singing voice, which I get not infrequently.
the singer's pitch and timbre change in a few spots too.
2️⃣ 𝐎𝐑𝐈𝐆𝐈𝐍𝐀𝐋 (synthwave J-Pop)
𝘧𝘶𝘭𝘭 𝘴𝘰𝘯𝘨: https://youtu.be/PYGtp0VLjS8
This one was made without ABC, just letting YuE2 compose a plan before generation.
a few dropped words, odd emphases in certain places, but mostly coherent
3️⃣ 𝑪𝑶𝑽𝑬𝑹 (*attempted*) German
進撃の巨人, vogel im kamfig YouSeeBigGirl:TT
(Reference: https://youtu.be/2o5H8wIR8Go)
I didn't get a decent generation for the song.
but, there was *one* this one section of the song 👏🏻
This happens; you'll have a killer track and *suddenly* some dude starts whispering pidgin language in a weird hermaphroditic AI voice for 2-3 sec, and then the song continues fine, no more cursed artifacts 😕
or, gen is a total trainwreck, but suddenly becomes coherent and makes something cool, like this thing 😌
m-a-p/YuE2-3B
Samples from other models:
- MiniMaxAI/MiniMax-Music3 : https://huggingface.co/posts/evdcush/304645800764284
- ACE-Step/acestep-v15-xl-sft : https://huggingface.co/posts/evdcush/646521189747912
natolambert
authored 10
papers 14 days ago
RewardBench 2: Advancing Reward Model Evaluation
Paper • 2506.01937 • Published • 7
Spurious Rewards: Rethinking Training Signals in RLVR
Paper • 2506.10947 • Published • 2
Generalizing Verifiable Instruction Following
Paper • 2507.02833 • Published • 1
Open Character Training: Shaping the Persona of AI Assistants through Constitutional AI
Paper • 2511.01689 • Published • 5
Olmo 3
Paper • 2512.13961 • Published • 38
Meta-Reinforcement Learning with Self-Reflection for Agentic Search
Paper • 2603.11327 • Published • 12
Olmo Hybrid: From Theory to Practice and Back
Paper • 2604.03444 • Published
Tmax: A simple recipe for terminal agents
Paper • 2606.23321 • Published • 17
Learning to Solve Hard Problems in RL for LLMs by Never Giving Up
Paper • 2609.13443 • Published • 14
Context Language Models
Paper • 2609.37725 • Published • 47
AdinaY
submitted a
paper to Daily Papers 22 days ago
BlueNipples
posted an update 23 days ago
Post
140
Feels good to get the result I did on Anansi. Literally exactly what I hoped for, from the merge, which rarely happens. Makes the potato feel less potato-y.
EduardoPacheco
authored a
paper 28 days ago
Post
78
𝐌𝐢𝐧𝐢𝐌𝐚𝐱 𝐌𝐮𝐬𝐢𝐜 𝟑 𝐬𝐚𝐦𝐩𝐥𝐞𝐬
# ミニマックスミュージック３ サンプル
サンプルの歌は：
- https://youtu.be/krF3NGRmwGY
- https://youtu.be/LoYAkJRcuqU
もっと歌はYTに聞けます。
この音楽は
MiniMaxAI/MiniMax-Music3
のモデルで作ってしました。
歌詞は日本語。
キャプションは英語。
（英語の方がうまく行けました）
ヴォーカルの質はとてもいいと思う。
ミニマックスミュージック３の
一つの注意するところは：
このモデルが中華風の音楽にオーバーフィットされてあるかもしれません。
（二胡が勝手に入っちゃうがちがあります）
Stack:
- interface: https://github.com/evdcush/minimax-music3-jukebox
- caption writer: https://github.com/evdcush/minimax-music3-skill-erhu-purge
試してみて〜
# ミニマックスミュージック３ サンプル
サンプルの歌は：
- https://youtu.be/krF3NGRmwGY
- https://youtu.be/LoYAkJRcuqU
もっと歌はYTに聞けます。
この音楽は
MiniMaxAI/MiniMax-Music3
のモデルで作ってしました。
歌詞は日本語。
キャプションは英語。
（英語の方がうまく行けました）
ヴォーカルの質はとてもいいと思う。
ミニマックスミュージック３の
一つの注意するところは：
このモデルが中華風の音楽にオーバーフィットされてあるかもしれません。
（二胡が勝手に入っちゃうがちがあります）
Stack:
- interface: https://github.com/evdcush/minimax-music3-jukebox
- caption writer: https://github.com/evdcush/minimax-music3-skill-erhu-purge
試してみて〜
Post
100
Sample output from ACE-Step/acestep-v15-xl-sft
I had issues with the upstream ACE-Step1.5 pipeline, and seeing there were other reports of issues with the python pipeline, but not the Comfy implementation, I used the Comfy
Inference params:
Both "lyrics" and "tags" were defined, but the the vocals in the output were not properly generated (perhaps due to the low sampler cfg).
https://youtu.be/lFiOUvSCv3w
I had issues with the upstream ACE-Step1.5 pipeline, and seeing there were other reports of issues with the python pipeline, but not the Comfy implementation, I used the Comfy
audio_ace_step1_5_xl_sft workflow.Inference params:
"TextEncodeAceStepAudio1.5": {
"bpm": 96,
"duration": 260,
"timesignature": "4",
"language": "ja",
"keyscale": "F# minor",
"cfg_scale": 2,
"temperature": 0.85,
"top_p": 1,
"top_k": 0,
"min_p": 0
}
"KSampler": {
"steps": 50,
"cfg": 1,
"denoise": 1
}Both "lyrics" and "tags" were defined, but the the vocals in the output were not properly generated (perhaps due to the low sampler cfg).
https://youtu.be/lFiOUvSCv3w
question on how to join and etc
9
#49 opened 5 months ago
by 
simonko912
Post
4,298
🚨 I've just published Sentence Transformers v6.0, introducing MultiVectorEncoder: ColBERT-style late interaction models are now a fourth model type, for training, inference, and interpretation, alongside the dense, sparse, and reranker models! Details:
Where a regular embedding model compresses a whole text into one vector, a multi-vector model keeps one vector per token and scores query against document with the MaxSim operator. That preserves token-level matching information that a single vector has to average away. It is also the state of the art for visual document retrieval, where a text query is matched against page images directly, charts and tables included, with no OCR step in between.
Any PyLate, Stanford ColBERT, or ColPali checkpoint loads straight into the same familiar API: model.encode_query(), model.encode_document(), and model.similarity() just work, whether the documents are texts or page images.
Does it help? LightOn trained LateOn (multi-vector) and DenseOn (dense) on the same data with the same 149M ModernBERT backbone, and the multi-vector model wins on 9 of the 13 NanoBEIR datasets: 0.6868 vs 0.6764 mean NDCG@10. The price is a bigger index, and the new HierarchicalTokenPooling module halves it at roughly no retrieval cost.
Antoine Chaffin, Raphaël Sourty, and I wrote a blog post walking through multi-vector models in practice: loading the various checkpoint formats, encoding and scoring, plugging them into a search stack, running them on page images, and keeping the index affordable. Check it out if you want to get started, or just point your Agent to the URL: https://huggingface.co/blog/multi-vector-encoder
pip install sentence-transformers==6.0.0
Release notes: https://github.com/huggingface/sentence-transformers/releases/tag/v6.0.0
Where a regular embedding model compresses a whole text into one vector, a multi-vector model keeps one vector per token and scores query against document with the MaxSim operator. That preserves token-level matching information that a single vector has to average away. It is also the state of the art for visual document retrieval, where a text query is matched against page images directly, charts and tables included, with no OCR step in between.
Any PyLate, Stanford ColBERT, or ColPali checkpoint loads straight into the same familiar API: model.encode_query(), model.encode_document(), and model.similarity() just work, whether the documents are texts or page images.
Does it help? LightOn trained LateOn (multi-vector) and DenseOn (dense) on the same data with the same 149M ModernBERT backbone, and the multi-vector model wins on 9 of the 13 NanoBEIR datasets: 0.6868 vs 0.6764 mean NDCG@10. The price is a bigger index, and the new HierarchicalTokenPooling module halves it at roughly no retrieval cost.
Antoine Chaffin, Raphaël Sourty, and I wrote a blog post walking through multi-vector models in practice: loading the various checkpoint formats, encoding and scoring, plugging them into a search stack, running them on page images, and keeping the index affordable. Check it out if you want to get started, or just point your Agent to the URL: https://huggingface.co/blog/multi-vector-encoder
pip install sentence-transformers==6.0.0
Release notes: https://github.com/huggingface/sentence-transformers/releases/tag/v6.0.0
lorraine2
authored a
paper about 2 months ago