**I am not a mathematician nor a professional coder. This was an experiment (with the help of AI of course).**
This is a custom-trained version of Google's Gemma 4 E4B intended for creative writing. It was trained using a custom implementation of **Distribution Fine-Tuning (DFT)** designed to mathematically penalize and eliminate repetitive AI slop and predictable phrasing.
The core training algorithm was inspired by the concepts outlined in the May 18, 2026 blog post, [*Fixing LLM Writing with Distribution Fine-Tuning*](https://rosmine.ai/2026/05/18/fixing-llm-writing-with-distribution-fine-tuning/).
Standard Supervised Fine-Tuning (SFT) and RLHF often cause models to regress to a generic, hyper-structured average of human text. To counter this, this model was trained by injecting a macro-statistical loss penalty into the backpropagation loop. By calculating the Mean Squared Error (MSE) between the model's batch-level vocabulary distribution and a human target distribution (or a good creative dataset), the model was actively penalized for overusing AI-frequent vocabulary (e.g., "whisper", "shiver", "sheer").
The model was trained on 1.3~ Epochs and effective batch size of 96, using a mix of multiturn roleplaying and creative writing dataset.
Credits to Rosmine and Google Gemini for the idea and the implementation. Let me know what you think in the Community section!
