Training supported by MS-SWIFT framework
#18
by tastelikefeet - opened
Check these scripts: https://github.com/modelscope/ms-swift/tree/main/examples/train/embedding
And with model-id:
...
--model google/embeddinggemma-300m \
--use_hf true \
...
or with modelscope:
...
--model google/embeddinggemma-300m \
--use_hf false \
...
We support multiple loss types, cosine-similarity, contrastive, infonce, etc.
Embedding training instructions can be found here:
https://swift.readthedocs.io/en/latest/BestPractices/Embedding.html
tastelikefeet changed discussion title from Training supported with MS-SWIFT framework to Training supported by MS-SWIFT framework
Hi @tastelikefeet ,
Welcome to Gemma family of open source models, thanks for your contribution. I really appreciates that your contribution to run the model with provided script. I would loike to try to run the model with above provided script.
Thanks.
MS-SWIFT does not support Matryoshka Representation Learning (MRL) loss?