magiccodingman
/

Jasper-Token-Compression-600M-ONNX-INT4

Sentence Similarity

sentence-transformers

feature-extraction

text-embeddings

text-embeddings-inference

Model card Files Files and versions

Jasper-Token-Compression-600M-ONNX-INT4 / README.md

magiccodingman's picture

Upload folder using huggingface_hub

879aca3 verified 2 days ago

|

History Blame Contribute Delete

1.04 kB

	---
	license: apache-2.0
	language:
	- en
	- zh
	library_name: sentence-transformers
	pipeline_tag: sentence-similarity
	base_model: infgrad/Jasper-Token-Compression-600M
	tags:
	- sentence-transformers
	- feature-extraction
	- sentence-similarity
	- text-embeddings
	- embeddings
	- retrieval
	- mteb
	- qwen3
	- onnx
	- onnxruntime
	- cpu
	- int-4
	---

	# Jasper Token Compression 600M — ONNX INT-4

	ONNX export of [infgrad/Jasper-Token-Compression-600M](https://huggingface.co/infgrad/Jasper-Token-Compression-600M).

	Precision: INT4
	Quantization: INT4
	Model size: 317.15 MiB

	INT4 ONNX export intended for highly compact CPU/local deployment while retaining the original model's 2048-dimensional embeddings.

	## Benchmarks

	\| Tokens \| Median latency \| Tokens/s \|
	\|---:\|---:\|---:\|
	\| 32 \| 57.347 ms \| 558.0 \|
	\| 128 \| 65.343 ms \| 1,958.9 \|
	\| 512 \| 96.749 ms \| 5,292.0 \|
	\| 1024 \| 137.314 ms \| 7,457.4 \|

	## Fidelity

	Median cosine similarity versus FP32: ~0.981–0.986.

	## Attribution

	Original model: `infgrad/Jasper-Token-Compression-600M`