nomic-ai

/

nomic-embed-text-v1.5

like 807

Follow

Nomic AI 1.09k

How can I infer with different embedding dimension?

model_path = "./Nomic_Embed_V1.5/onnx/model_quantized.onnx"
tokenizer_path = "./Nomic_Embed_V1.5"
tokenizer = AutoTokenizer.from_pretrained(tokenizer_path)
session = ort.InferenceSession(model_path)
inputs = tokenizer(query_sentence, return_tensors="np")

onnx_inputs = {session.get_inputs()[i].name: inputs[input_name].astype(
np.int64) for i, input_name in enumerate(inputs.keys())}
outputs = session.run(None, onnx_inputs)
embeddings = outputs[0]

I don't believe the model was converted to Onnx with that ability. You will have to do that yourself unfortunately

Thanks. Already working on one

Let me know if you get something working!