Devstral?
Should try this on devstral. It's updated and mostly like large but with additional knowledge and agentic training.
I've considered both the 24b and 123b devstral. I do worry that finetuning or heretic will likely cause significant damage to these specialized models.
Did you use le-wrong template for this one? Mistral didn't start the [system prompt] until the 25xx version. And I thought this was only heretic'd. I added it to the download queue and it popped into my head later on. I know I'm using the wrong template when it can't complete a specifically formatted character card correctly. If you finetuned after on that format it would likely do a fixup. I will try both and see how it goes. Usually mistrals aren't very censored for me at all so I'm curious to compare.
On UGI there was only slight damage from whatever you did to natint and a couple other categories. There's also deslop in heretic now for added fun. Devstral big loves to start messages with "oh" to the point I banned the token. Might help with that.