Jasper Token Compression 600M — ONNX INT-4
ONNX export of infgrad/Jasper-Token-Compression-600M.
Precision: INT4
Quantization: INT4
Model size: 317.15 MiB
INT4 ONNX export intended for highly compact CPU/local deployment while retaining the original model's 2048-dimensional embeddings.
Benchmarks
| Tokens | Median latency | Tokens/s |
|---|---|---|
| 32 | 57.347 ms | 558.0 |
| 128 | 65.343 ms | 1,958.9 |
| 512 | 96.749 ms | 5,292.0 |
| 1024 | 137.314 ms | 7,457.4 |
Fidelity
Median cosine similarity versus FP32: ~0.981–0.986.
Attribution
Original model: infgrad/Jasper-Token-Compression-600M
- Downloads last month
- -
Model tree for magiccodingman/Jasper-Token-Compression-600M-ONNX-INT4
Base model
Qwen/Qwen3-0.6B-Base Finetuned
Qwen/Qwen3-Embedding-0.6B
from sentence_transformers import SentenceTransformer model = SentenceTransformer("magiccodingman/Jasper-Token-Compression-600M-ONNX-INT4", trust_remote_code=True) sentences = [ "That is a happy person", "That is a happy dog", "That is a very happy person", "Today is a sunny day" ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [4, 4]