Unable to use the model with text-embeddings-inference
#1
by yco - opened
The model seem to report the variant of the model as 'bilingual' instead of 'xlm-roberta', which causes text-embeddings-inference (https://github.com/huggingface/text-embeddings-inference) to refuse to use the model.
Is this motivated by an architecture change ?
To reproduce
model=Lajavaness/bilingual-embedding-base
volume=$PWD/data
docker run --platform linux/x86_64 -p 8080:80 -v $volume:/data --pull always ghcr.io/huggingface/text-embeddings-inference:cpu-1.4 --model-id $model
Output
2024-07-05T09:55:48.701830Z INFO text_embeddings_router: router/src/lib.rs:250: Starting model backend
Error: Could not create backend
Caused by:
Could not start backend: Model is not supported
Caused by:
unknown variant `bilingual`, expected one of `bert`, `xlm-roberta`, `camembert`, `roberta`, `distilbert`, `nomic_bert`, `mistral`, `new`, `qwen2` at line 6 column 27
Update: changing only "model_type": "xlm-roberta", in config.json makes it compatible with text_emeddings_inference