Chroma.v1-HD + T5XXL Unchained = Genuine Freedom
Lodestones, thanks so very much for this foundational model!
I know it hasn't quite taken off like we'd all hoped, but I suspect its a simple matter of exposure and time. That and a single last hurdle to true Open Source freedom of choice. A model can be all inclusive and unbiased, but if it uses a translator that doesn't understand a concept, it'll require a LOT more description to achieve it. So- I was looking into ways to improve the Tokenization of advanced NSFW concepts; cuz I'm a 'sucker for tentacles', (pausing for groan over pun)..
Anyway, I came across Kaoru8's T5XXL Unchained Tokenizer ( https://github.com/Kaoru8/T5XXL-Unchained ), that among other things more than doubled the vocabulary by adding in the exact list Google used to Leave Out of the T5XXL encoder, and then also adding selected Danbooru tags, 200+ nationalities, Character names, ... a lot.
With the patch given to use it in ComfyUI, I can report fantastic success. I'm currently working with ChatGPT-Pro to set up some runpod VM's to tackle a full fine tune of Chroma1-HD with the Unchained T5XXL Tokenizer. But if its something that catches your interest, and since I have no clue what I'm doing, let us all know? :D
@NeonScreams
will need a custom loader
https://github.com/AbstractEyes/comfy-clip-shunts/tree/dev2_electric_boogaloo doesn't support Chroma
@NeonScreams will need a custom loader
https://github.com/AbstractEyes/comfy-clip-shunts/tree/dev2_electric_boogaloo doesn't support Chroma
I'm not familiar with that Node for ComfyUI.
Also not quite sure how it relates to what I mentioned?
I'm using the default ComfyUI "Load Clip" node shown below in the screenshot.
Hope that helps!
i read about the git repo.
seems we still need to train t5xxl for those extended concept.
if we dont train that, new stock t5xxl-unchained SHOULD NOT know about the "meaning" of extended tokens.
so, you are getting improvement post training? or just use the model in https://huggingface.co/Kaoru8/T5XXL-Unchained?
You are correct - The extended tokens and concepts will not be represented by adherence unless retrained.
However, the known tokens should show improved adherence to terms. Unless it’s confirmation bias on my part? I can only say I have noticed improved adherence within my prompts for NSFW Tokenization.
But the idea was to ensure users could feel comfortable in the short and long term for using the Unchained version. We have access to generation within Chroma immediately without completing the Fine-Tune. With the promise of better adherence after reworking the model.
(I might need to edit to say I’m using this within ComfyUI for local generation, not on a site)