There is another VBVR
there is another VBVR for LTX in civitai it has this info
V3 has been optimized for motion. Expect noticeably livelier, more dynamic movement compared to previous versions.
What changed in V3: Attention-only layers. The feedforward layers have been stripped, leaving only the attention weights. It seems like the prompt following and reasoning behavior most likely live in the attention layers, while the feedforward layers were potentially interfering with natural motion, likely by over-learning features like textures and style from the training data.
is it the same implementation ?
I didn't test training without FFN,but that is an interesting idea.
Somehow both at civit.red and this one performs by a big margin worse than official one vbvr :D
I didn't test training without FFN,but that is an interesting idea.
did you train the recent 240K release without the FFN or stick to previous process?
I didn't test training without FFN,but that is an interesting idea.
did you train the recent 240K release without the FFN or stick to previous process?
nope, this will happen in stage 3 training, I analyzed the dataset and found that certain tasks require acting on the FFN layers. So in Stage 3, I extracted 150K high-difficulty samples that are unrelated to FFN for reinforcement training.