Hugging Face's logo

jinaai
/
xlm-roberta-flash-implementation

Transformers
xlm-roberta
🇪🇺 Region: EU
Model card Files
xet
 Community
60

support lora

#1
by jupyterjazz - opened
base: refs/heads/main
←
from: refs/pr/1
Discussion Files changed
+784
-15
jupyterjazz
 Jina AI org
•
edited Apr 17, 2024

https://github.com/jina-ai/finetuner-core/issues/926
feat: support lora5ed05aac
 Update modeling_lora.pyc380b5a6
 feat: initialize models with or without adapters79c3c939
 chore: change num lora def valuef9601151
jupyterjazz changed pull request status to open
small change6060bad3
michael-guenther
 Jina AI org

Looks good! Some questions:

  • Do we need another class to define in the automap of the config?
  • What is this add_pooling_layer in the initialization?
  • Does it make sense to change num_loras=1 by default or is 0 better?
  • Maybe there are a few other things which might make sense to set in the config, e.g. the rank
feat: merge stuff841b70fc
jupyterjazz changed pull request status to merged

Sign up or log in to comment