magiccodingman
/

Jasper-Token-Compression-600M-ONNX-INT8

Sentence Similarity

sentence-transformers

feature-extraction

text-embeddings

text-embeddings-inference

Model card Files Files and versions

Jasper-Token-Compression-600M-ONNX-INT8 / README.md

magiccodingman's picture

Upload folder using huggingface_hub

16121ff verified 2 days ago

|

History Blame Contribute Delete

1.02 kB

	---
	license: apache-2.0
	language:
	- en
	- zh
	library_name: sentence-transformers
	pipeline_tag: sentence-similarity
	base_model: infgrad/Jasper-Token-Compression-600M
	tags:
	- sentence-transformers
	- feature-extraction
	- sentence-similarity
	- text-embeddings
	- embeddings
	- retrieval
	- mteb
	- qwen3
	- onnx
	- onnxruntime
	- cpu
	- int-8
	---

	# Jasper Token Compression 600M — ONNX INT-8

	ONNX export of [infgrad/Jasper-Token-Compression-600M](https://huggingface.co/infgrad/Jasper-Token-Compression-600M).

	Precision: INT8 (Dynamic)
	Quantization: Dynamic INT8
	Model size: 583.56 MiB

	Dynamic INT8 ONNX export optimized for fast CPU inference. This is a text embedding model.

	## Benchmarks

	\| Tokens \| Median latency \| Tokens/s \|
	\|---:\|---:\|---:\|
	\| 32 \| 44.362 ms \| 721.3 \|
	\| 128 \| 48.609 ms \| 2,633.3 \|
	\| 512 \| 63.180 ms \| 8,103.8 \|
	\| 1024 \| 84.619 ms \| 12,101.3 \|

	## Fidelity

	Median cosine similarity versus FP32: ~0.988–0.992.

	## Attribution

	Original model: `infgrad/Jasper-Token-Compression-600M`