LLMWildling

/

gemma-4-120b-a12b-coder

like 6

@lordfervi ive been going at this for 2 years now, its pretty complicated and I still work on perfecting it. do you want me to make you one? feedback is super welcome. enjoy.

@LLMWildling
Let's just say this, we'll stay in touch :)

People usually do various types of model optimization. Your models look like they do some fine-tuning.

I'm afraid we'll reach a point where you'll have enterprise models in the cloud (OpenAI, Claude), some large open-source models (like DeepSeek), but not for consumers (too expensive), you'll have small models (like Gemma), but very few mid-tier models (like Mistral Small 4, GPT-OSS, etc.).

If it's possible to "easily" improve AI models, I think it's revolutionary.

I accidentally found your profile and see that you've made a lot of models much larger. I'm wondering if it works correctly and so on.

Unfortunately, I don't have the DGX Spark for now (I hope I'll have it next week). I'll let you know when I can test it.

Maybe in the future you'll manage to slightly (even not significantly) tune Mistral Small 4 ;)

I just checked my pipeline and but pretty sure it's pre/post training, the full run. I use my own optimizer to speed things up.

"If it's possible to "easily" improve AI models, I think it's revolutionary." - thats the idea, but things take time to perfect. so any feedback from this community is welcome, I have a lot more coming.

Mistral Small 4 - do you want a bigger version? @lordfevi

@LLMWildling

That's interesting.

I don't need it right now, but once I get the machine, I'll test the current models :)

It's just that what you're doing is interesting, and that's why I'm asking :)

@LLMWildling

I have a DGX Spark for testing.

I have an alternative question. Can you try creating powerful models on DiffusionGemma?

It’s Gemma 4, which works on the principle of diffusion. In short—it’s worse (in benchmarks), but much faster. It’s more of a demo, really.

I’ll be checking out your models, but the DGX Spark is very slow over Wi-Fi.

@lordfervi its possible it depends on if there is support in transformers for it. its possible. Currently I am stuck in quantization hell, im loosing accuracy going from bf16 to mxfp4/nvfp4 . im trying to fix that first and I can take a look at it.

I just had a ok run with nvfp4. let me see what I can do @lordfevi

@lordfevi its time. how big of a DiffusionGemma do you want? do you have your dgx yet? I stand ready my lord.

Im thinking maybe we target around 100b. but do a a dense model like 30b-50b? because this inference is 4x faster it may work faster.

@LLMWildling

For now, let's run a demo to see if everything works :)

I have a DGX Spark. However, I see a problem: while LM Studio theoretically allows you to run really large models, vLLM has trouble with that. I’m currently using Qwen3.6 27B, and it’s working surprisingly well.

Another thing I’m curious about: Is it possible to connect such a large model to DFlash, or will it not work—or at least not work well? If you know anything about it.

yes! of course. vllm is the way to go to get the most value out of your box, if you plan on running multiple agents.

All of my models are mostly optimized for fused inference like vllm. What do you see or do when vllm needs to be run? show me your errors

I just made some new nemotrons if you are interested:
https://huggingface.co/LLMWildling/Nemotron-175b-A13b-Coder-NVFP4
https://huggingface.co/LLMWildling/NVIDIA-Nemotron-3-Super-165B-A13B-Coder-NVFP4
https://huggingface.co/LLMWildling/NVIDIA-Nemotron-3-Super-155B-A13B-Coder-NVFP4
https://huggingface.co/LLMWildling/NVIDIA-Nemotron-3-Super-145B-A13B-Coder-NVFP4
https://huggingface.co/LLMWildling/NVIDIA-Nemotron-3-Super-135B-A13B-Coder-NVFP4

@LLMWildling

(EngineCore pid=201) ERROR 07-14 22:49:13 [core.py:1231] OSError: Can't load feature extractor for 'LLMWildling/gemma-4-100b-a10b-coder'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'LLMWildling/gemma-4-100b-a10b-coder' is the correct path to a directory containing a preprocessor_config.json file

It's about Gemma 100b a10b coder.

@lordfervi im working on a v2 of that coder. itll be a bit better and more tuned for open code usage. should hopefully have something for you today/tomorrow?

@LLMWildling sorry for hacking the thread -- do you have bf16 for gemma4 120b 12b or similar weight model -- i am asking this because i have a old server with 2x rtx3090 and 2x e5-2680v4 with 16x 64gb ddr4-2400 ecc -- i am thinking to create a good mixed quantized version of this or something similar model with ik llama cpp -- so that i can run agentic /code work personally !! i tried your nemotron 175b 13b and tried to quantise it but it is already 4 bit quantised so it is not a good base , i also tried gemma4 52b but that too is 4 bit qunatised -- btw awesome work by adding these models for community !!

hey @pawarshardul thank you for kind words! Believe it or not one of my biggest constraints is space :/ I build/tune 24/7 so to make space for new experiments I delete the old ones. I dont have the bf16 weights anymore...

However I am currently working on a new training technique. Pre/post training the gemma models I discovered some possible model collapses in certain tokenizer scenarios. Also google recently made some changes that caught my attention. So im currently working on a very special Gemma 4 OpenCode version. I specifically post trained and RL'd this one in the OpenCode harness. I should have these weights in bf16 if everything goes right. In my evals it was able to oneshot a pretty good angry birds clone with full physics.

hey @pawarshardul thank you for kind words! Believe it or not one of my biggest constraints is space :/ I build/tune 24/7 so to make space for new experiments I delete the old ones. I dont have the bf16 weights anymore...

However I am currently working on a new training technique. Pre/post training the gemma models I discovered some possible model collapses in certain tokenizer scenarios. Also google recently made some changes that caught my attention. So im currently working on a very special Gemma 4 OpenCode version. I specifically post trained and RL'd this one in the OpenCode harness. I should have these weights in bf16 if everything goes right. In my evals it was able to oneshot a pretty good angry birds clone with full physics.

wow!!! Cant wait that to try out!!! Btw ever think of making big moe of qwen 3.6 -- its dense model 27b is currently being praised by many !!! But MOE model doesnt get that much praise -- https://huggingface.co/migtissera/Tess-4-35B-A3B look into this -- quite good model in my testing -- anyway -- i love gemma models -- the dense one is able make pinescripts with very minimal errors -- and i found it better than qwen models in this regards .
also i made gguf out of your quantized source of

gemma-4-180b-a22b-coder-mxfp4 and tools calling works ok -- i guess -- i also made ggufs of nemo 175b coder and gptoss220ba20b -- yet to test them -- my hardware is does not allow me to fully load to gpus so my i had to use gpu+cpu+ram loading -- so token gen is slower -- but i absolutely love running these local llms -- it is better than using expensive subscriptions -- so i take help of gemini/grok if i stuck with local llms

@pawarshardul I hope you are doing well. I have something experimental for you, it's also my first bf16 release so let me know how it is. I named them after you.
going to be doing a bit more tuning over the next couple of days.
https://huggingface.co/LLMWildling/gemma-4-pawarshardul-opencoder-48b-a10b-bf16
https://huggingface.co/LLMWildling/gemma-4-pawarshardul-opencoder-40b-a8b-bf16

oh yeah... these are very opencode specific and you'll get a big boost using them in there.

Incredible!! thank you for models -- and thank you for making them in my name (although that is strictly not needed) -- i will test out and get back to you!!

I got another potential drop later today :D

Can you go into more detail on how you’re making larger versions of the models? Do you start by duplicating tensors that exist, then do more training?

@mrebersv I have my own optimizer. I dont use Adam 8bit. I train much faster at a lower quant.