cross-encoder/ms-marco-TinyBERT-L-2-v2 not returning list of scores
Hello, When invoking this model for a list of ('Query', 'Paragraph1') I am expecting to get a list of scores as output:
For example,
from sentence_transformers import CrossEncoder
model = CrossEncoder("cross-encoder/ms-marco-TinyBERT-L-2-v2", max_length=512)
scores = model.predict(
[('How many people live in Berlin?',
'Berlin had a population of 3,520,031 registered inhabitants in an area of 891.82 square kilometers.'),
('How many people live in Berlin?', 'Berlin is well known for its museums.')]
)
print("scores:", scores)
scores: [ 7.1523647 -6.2870436]
This is what I get while running in Python code.
But while running thru Inference API on the Model Card page, as well as in Sagemaker deployment I just get one score, as an output:
Sagemaker deploy params used:
HF_MODEL_ID: cross-encoder/ms-marco-TinyBERT-L-2-v2
HF_TASK: text-classification
Output:
[
[
{
"label": "LABEL_0",
"score": 0.00012489620712585747
}
]
]
Anything I am missing? How can I get a list of scores as an output (In Sagemaker deployment), in above example?