Ideas

#112
by HappyHugger - opened

How are the captions being made for the dataset? Is it Qwen? or human captioning? I think human captioning is far more powerful than using ai to auto-caption. If Qwen captions a video it usually leaves out the actions and describes one frame of the video in my experience using fizgig to create Loras. If a man is sitting at a bench with shrubs or bushes behind him, stands up and walks to the left of the frame, Qwen will say a blond man sitting on a bench, describe his clothing, with a blurry background. It will leave out the action. Also, it seems that it was trained on up-close videos of people only, as the camera zooms out and away, the faces lose integrity like it never saw that before. It should be trained on individuals (and maybe groups) approaching the camera, and up-close videos of people where the camera zooms out and away, or people walking backwards. I feel like this careful engineering could improve the model greatly. It could also help to train it on individuals who leave the frame and return, so that it can remember faces better (only a guess, I'm not really sure that would work)

Sign up or log in to comment