[Solved] "Undertrained" Loras
Hi! I would like to draw attention to an issue that has already been raised on Reddit. It seems that some (hopefully not all) Loras are not sufficiently trained. I have already been able to confirm this with a few Loras. It seems that the further away they are, the less recognizable some characters become as they approach the base checkpoint. For example, the character becomes an Asian character. This circumstance then disappears again in portraits. To support my thesis, I used the same prompt to generate an image of Jennifer Aniston with your Lora and an image with Malcom's Lora. I think the differences are obvious.
Malcom:
nphSi
Can you please share the prompt and settings so i can reproduce this?
Can you please share the prompt and settings so i can reproduce this?
Sure buddy,
photo realistic, high quality photo, realistic, full length photograph, 16k, HDR, uber detailed.
A woman is wearing a pink long-sleeved high-collar fuzzy sweater, red with black dots flannel mini-skirt and red stiletto strappy sandals. She has gold earrings and carries a red leather purse. She is on the balcony of a high rise Manhattan apartment building looking out at night over Times Square. The balcony has glass walls. Ultra-detailed environment, natural textures,
vibrant lighting, clean architectural lines.
Euler Beta, 8 Steps, CFG 1
You need to use the full lora name in the prompt. I caption in this way to reduce character bleeding/merging. Try this:
photo realistic, high quality photo, realistic, full length photograph, 16k, HDR, uber detailed.
Jennifer Aniston (vrtlJenniferAniston) is wearing a pink long-sleeved high-collar fuzzy sweater, red with black dots flannel mini-skirt and red stiletto strappy sandals. She has gold earrings and carries a red leather purse. She is on the balcony of a high rise Manhattan apartment building looking out at night over Times Square. The balcony has glass walls. Ultra-detailed environment, natural textures,
vibrant lighting, clean architectural lines.
Ahh! Much better. Thanks for the info. Does that apply to all Loras? So first and last name? And what does that mean in brackets: vrtlJenniferAniston (vrtl)
(vrtlxxxxx) is my unique token for every lora i make. Everything the model learns is associated with that token. The base model itself knows a bit about i.e. Jennifer Aniston, so using Jennifer Aniston (vrtlJenniferAniston) blends base model knowledge and lora knowledge together while "Woman" still is the standard asian of the z-image checkpoint.
My version:
Frontal photo of Jennifer Aniston (vrtlJenniferAniston) wearing pink cashmere pullover and red tartan patterned mini skirt, red high heel sandals, red handbag, standing on the balcony in the fifth floor of a skyscraper, a city with skyscrapers at dark night in the background, looking down at the street, neon signs and steamy air
(vrtlxxxxx) is my unique token for every lora i make. Everything the model learns is associated with that token. The base model itself knows a bit about i.e. Jennifer Aniston, so using Jennifer Aniston (vrtlJenniferAniston) blends base model knowledge and lora knowledge together while "Woman" still is the standard asian of the z-image checkpoint.
So with token you mean "trigger" word or is it something else? Just curious as I'm training loras myself using the yaml file from malcom. I often have the problem that if there are more than 2 people in the image, the lora start bleeding which means for me that the second person looks like the lora character. Is that what you meant? I always thought this was something that couldn't be changed. Or do you mean something else with bleeding?
Thats exactly what i mean. And yes token means trigger word. Its not a 100% fix for the same face problem but it helps. The models are not well trained for different unique people interacting. Good prompting can help too. Current models really really want it described precisely.