maxiboch/MiA-Emb-8B-onnx

MiA-Emb-8B ONNX

ONNX conversion of MindscapeRAG/MiA-Emb-8B for fast CPU/GPU inference.

Model Info

• Parameters: 8B
• Embedding Dimension: 4096
• Max Sequence Length: 8192

Usage with ONNX Runtime

import onnxruntime as ort
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("maxiboch/MiA-Emb-8B-onnx")
session = ort.InferenceSession("model.onnx", providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])

inputs = tokenizer("Your text here", return_tensors="np", padding=True, truncation=True)
outputs = session.run(None, dict(inputs))
embeddings = outputs[0]

Conversion

Converted to ONNX by @maxiboch.

Original Model

• Source: MindscapeRAG/MiA-Emb-8B
• License: Apache 2.0