--- base_model: coder3101/gemma-4-E4B-it-heretic tags: - text-generation-inference - transformers - unsloth - gemma4 license: apache-2.0 language: - en --- [](https://github.com/unslothai/unsloth) *goated software^^* # DISCLAIMER **I am not a mathematician nor a professional coder. This was an experiment (with the help of AI of course).** This is a custom-trained version of Google's Gemma 4 E4B intended for creative writing. It was trained using a custom implementation of **Distribution Fine-Tuning (DFT)** designed to mathematically penalize and eliminate repetitive AI slop and predictable phrasing. The core training algorithm was inspired by the concepts outlined in the May 18, 2026 blog post, [*Fixing LLM Writing with Distribution Fine-Tuning*](https://rosmine.ai/2026/05/18/fixing-llm-writing-with-distribution-fine-tuning/). Standard Supervised Fine-Tuning (SFT) and RLHF often cause models to regress to a generic, hyper-structured average of human text. To counter this, this model was trained by injecting a macro-statistical loss penalty into the backpropagation loop. By calculating the Mean Squared Error (MSE) between the model's batch-level vocabulary distribution and a human target distribution (or a good creative dataset), the model was actively penalized for overusing AI-frequent vocabulary (e.g., "whisper", "shiver", "sheer"). The model was trained on 1.3~ Epochs and effective batch size of 96, using a mix of multiturn roleplaying and creative writing dataset. Credits to Rosmine and Google Gemini for the idea and the implementation. Let me know what you think in the Community section!