Gryphe

/

Gemma-4-26B-A4B-StyleTune

like 47

could you just fix this when i try mlx it?
ValueError: Received 1 parameters not in model: lm_head.weight.

also same issue with 31B version....

Makes sense, considering this model has an extra tensor that others won't have! It's duplicated from the original embed_tokens, then trained in isolation.

Not an awful lot I can do about that, sorry!

I’m absolutely thrilled with this fine-tune. I tried it out for the first time today and was blown away. I knew Gemma was good at following instructions, but I’ve never seen it handle them quite this well. I asked it to incorporate natural spelling errors, repeated letters, lowercase text, and so on. At first, I was wondering what was going on—why it was writing so "weirdly"—until I realized it was simply following my system prompt :D It feels refreshing—different. You can have really deep conversations with it. I like that. The last time I was feeling this was when I was using Mistral Nemo.

Noticeably better writing than other Gemma4 models I've tried, but after a decently long context, it starts getting rather unstable in Silly Tavern. Eventually it was devolving into gibberish after its responses regularly, and I had to switch back to using G4-Meromero.

It's very possible this style tune needs completely different sampler settings since it generally has more tokens to choose from then it ever did before - Once a strange token manages to sneak in, events tend to... escalate.

Some feedback.

1. I'm playing in a non-Latin language and noticed that the model (like Mistral Nemo) is writing some words in English (which neither the original Gemma 4 nor the finetuning I've tried does). I added an instruction to the author's note that forces the model to respond in the same language I'm writing in, but over time, the model started writing strange, awkward words. I see the "English" tag, but the model's profile says that other languages ​​are unaffected—apparently, this is not the case.
2. The model prefers to write in second person and ignores the author's note that says "Write in third person." Moreover, she switched to second-person narration right in the middle of a session, and regeneration was unable to return her to third-person. Because of this, the model started describing what I see (I have restrictions against describing {{user}}'s words, thoughts, or actions, but there was no restriction against describing what they see, so it found a loophole).
3. I test different models on the same romantic comedy scenario (this is explicitly stated in the system prompt), and I'm well aware of how {{char}} behaves under a variety of models. I made two attempts, and both were not entirely romantic – in one, the character simply left (my system prompt allows this, giving characters the freedom to act according to their motives), and in the other, the interaction was not particularly romantic. Either I rolled the "unromantic root" twice in a row, or the training data lacked romance, and the model overfitted and began avoiding it.
4. Otherwise, the model achieved its intended goal – I really didn't encounter any of G4's favorite cliches, and overall, it felt like a different experience. If not for the issues described above, this would be the best FineTune G4.

In any case, thank you for your work. I hope this review is useful for your future projects.

P.S. It would be great if you could provide this tensor as a LoRA tensor – I'm using a custom quant, and with LoRA I could connect your StyleTune to my quant.

I was thinking about this again this morning, and I'm wondering if it's possible that the finetuning is interfering with the beginning of string and end of string tokens. I noticed that this model tends to give me longer than usual outputs, and the gibberish usually starts after a usual length response has been given. At one point it just filled all the remaining tokens with blank spaces. One workaround may be reducing the amount of tokens the llm is allowed to respond with if affected by this issue. I tend to have mine set much higher than normal because I prefer longer responses when possible.

I'm just a layman though, I don't know if those tokens are even affected by the way you've finetuned the model. I'm just hoping this helps since I really like the output of this model otherwise.

I know this isn't your problem just wanted to say
I used styleswap from https://huggingface.co/andyoneal/Dark-Scarlett-v1.0-26B-A4B-StyleSwap-GGUF to transplant a q8 version of this model's output.weight atop a q6_k https://huggingface.co/llmfan46/G4-MeroMero-26B-A4B-it-uncensored-heretic-GGUF
and it seems to work fairly well, though it doesn't seem to produce the reasoning traces by itself no matter the template or reasoning forced on ie: <|channel>thought
but it reasons normally if the reasoning stuff is forced via assistant preamble.

Edit: after messing with it, the transplant worked less good than originally thought.

I'm starting to suspect that MoE models need a more gentle approach to remain stable (The Qwen run taught me this) so if there's interest I'm totally willing to try and produce a v2 attempt for 26B-A4B.

Would you consider using a dataset with a mix of languages for non-English users? Currently I don't know if such a dataset exists on HF, but I'm sure one can be found.

@SaineGrey Limiting the length of a response often results in it being cut off mid-sentence. To control the length, use system instructions. Add something like this to the author's note:
[Maintain a tight narrative pace. Each response should focus on a single narrative beat — one meaningful sequence of actions or one exchange of dialogue. Avoid "action stacking." Do not perform multiple distinct or large-scale actions in a single turn. Do not advance the scene too far ahead of {{user}}. Aim for a response length that complements the user's input. Avoid excessive verbosity that overwhelms the interaction.]
This should be added to the author's note, otherwise the impact of the instructions will weaken as the context increases, and the model will gradually verbose the response. Ideally, the entire system prompt should be kept in the author's note — this way, the model will carefully reread it before each response.

@Gryphe The Gemma 4 is currently the most popular model on HF. The 26B-A4B ranks #5 among models <3B (ahead are only the outdated Qwen2.5/3, which have accumulated numerous downloads over time), and the 31B is in #6. While the 31B requires a GPU with at least 24 GB of VRAM, MoE can be run on 24 GB of RAM without a GPU at all and will have decent speed, while the quality difference is not that big. I don't understand ML, but judging by the KL div tests, the 26B-A4B is extremely fragile - even in Q6_K, its KL div is higher than the 31B's in Q4_K_M. It's likely that any careless move could easily break this model. But in any case, my vote goes to it.

i would very much like a v2 version of 26-A4b, there is not enough being done on this model, i think it has a lot of potential

Just so you're aware, https://huggingface.co/Gryphe/Gemma-4-26B-A4B-StyleTune-V2 is now a thing! We'll all have to wait a bit for GGUFs, but this version should be far, far more stable then this one.