Looks ....

#2
by tcclaviger - opened

Qwen3.8 Flash Next....that you?!

Shuffling the engram is cute.

For those interested in using, unshuffle the engram and run as a qwen4exp model with a small adjustment to handle the fused q/k/v // qk_v. identical vision, hc, etc weights, fine tune on experts, shuffled engram. Add the MTP back and it'll still do fairly well.

4 copied layers, engram shuffling, some obfuscation over the changes to make it just enough to not work on qwen4exp code, and then forces you to run the I'm sure totally and completely not nefarious custom execution code...

EDIT
Further looking, the extra 4 layers, yep, not even active, completely skipped by their custom code. Dead weight to differentiate slightly from Qwen3.8 Flash Next. Custom code also fetches from a remote at run time, use if you dare.

Actively blocks Qwen3.8 MTP head, clearly to make it so you can't just slap it on and reveal how little it's been changed.
At load time the shuffled engram is switch back to Qwen3.8 engram hash order, lmao.
qkv and qk_v, just to make it so their kernel knows it only, not at all odd to force people on to your kernel, totally legit practice...
The 256 extra tokenizer entries marked dormant...yeah right.

custom code is slopped together trash of multiple files glued into one .py file.

Model weights read identical to the 3rd or 4rd decimal to Qwen3.8 Flash Next weights.

Sign up or log in to comment