GrowBitLabs
/

tinye5

@@ -66,31 +66,40 @@ where model size, CPU latency, and deployment efficiency matter.
 | Model                           |        Size | STS Spearman ↑ | SciFact Recall@10 ↑ | SciFact nDCG@10 ↑ |
 | ------------------------------- | ----------: | -------------: | ------------------: | ----------------: |
-| **TinyE5-L6-384 (Safetensors)** |     86.7 MB |         0.8138 |              0.7148 |            0.5701 |
-| **TinyE5-L6-384 (FP32 ONNX)**   |     86.2 MB |         0.8138 |              0.7148 |            0.5701 |
-| **TinyE5-L6-384 (INT8 ONNX)**   | **21.8 MB** |         0.8070 |              0.7259 |            0.5791 |
-| E5-small-v2                     |      133 MB |     **0.8574** |          **0.8093** |        **0.6797** |
 ### CPU Inference Performance
-| Model                  | Precision |        Size |        Latency ↓ |      Throughput ↑ |
-| ---------------------- | --------- | ----------: | ---------------: | ----------------: |
-| TinyE5-L6-384 ONNX     | FP32      |     86.2 MB |     3.65 ms/text |     274.3 texts/s |
-| **TinyE5-L6-384 ONNX** | **INT8**  | **21.8 MB** | **2.65 ms/text** | **377.7 texts/s** |
 ### INT8 vs FP32 ONNX
-| Metric            |          FP32 |              INT8 |           Change |
-| ----------------- | ------------: | ----------------: | ---------------: |
-| Model size        |       86.2 MB |       **21.8 MB** | **~75% smaller** |
-| CPU latency       |  3.65 ms/text |  **2.65 ms/text** |   **~27% lower** |
-| CPU throughput    | 274.3 texts/s | **377.7 texts/s** |  **~38% higher** |
-| STS Spearman      |    **0.8138** |            0.8070 |          -0.0068 |
-| SciFact Recall@10 |        0.7148 |        **0.7259** |          +0.0111 |
-| SciFact nDCG@10   |        0.5701 |        **0.5791** |          +0.0090 |
-> **Summary:** INT8 quantization reduces TinyE5-L6-384 to just **21.8 MB** while increasing CPU throughput to approximately **378 texts/s**, with only a small change in semantic similarity performance and no degradation on the tested SciFact retrieval benchmark.
 ## Docker Example
@@ -106,7 +115,7 @@ services:
       - ./data:/data
     command:
       - --model-id
-      - rizwan3d/tinye5
       - --revision
       - int8-onnx
       - --pooling
@@ -118,7 +127,7 @@ services:
 All variants are published under the same Hugging Face repository:
 ```text
-rizwan3d/tinye5
 ```
 | Variant          | Revision    | Model               |
@@ -132,7 +141,7 @@ rizwan3d/tinye5
 ```yaml
 command:
   - --model-id
-  - rizwan3d/tinye5
   - --revision
   - main
   - --pooling
@@ -144,7 +153,7 @@ command:
 ```yaml
 command:
   - --model-id
-  - rizwan3d/tinye5
   - --revision
   - fp32-onnx
   - --pooling
@@ -156,7 +165,7 @@ command:
 ```yaml
 command:
   - --model-id
-  - rizwan3d/tinye5
   - --revision
   - int8-onnx
   - --pooling

 | Model                           |        Size | STS Spearman ↑ | SciFact Recall@10 ↑ | SciFact nDCG@10 ↑ |
 | ------------------------------- | ----------: | -------------: | ------------------: | ----------------: |
+| **TinyE5-L6-384 (Safetensors)** |     86.7 MB |         0.8138 |              0.7342 |            0.5897 |
+| **TinyE5-L6-384 (FP32 ONNX)**   |     86.2 MB |         0.8138 |              0.7342 |            0.5897 |
+| **TinyE5-L6-384 (INT8 ONNX)**   | **21.8 MB** |         0.8058 |              0.7599 |            0.6020 |
+| all-MiniLM-L6-v2                |           — |     **0.8194** |          **0.7923** |        **0.6494** |
 ### CPU Inference Performance
+| Model                  | Precision |        Size |        Latency ↓ |       Throughput ↑ | Session RAM ↓ |
+| ---------------------- | --------- | ----------: | ---------------: | -----------------: | ------------: |
+| TinyE5-L6-384 ONNX     | FP32      |     86.2 MB |     2.35 ms/text |      424.7 texts/s |       90.4 MB |
+| **TinyE5-L6-384 ONNX** | **INT8**  | **21.8 MB** | **0.77 ms/text** | **1299.4 texts/s** |   **11.2 MB** |
 ### INT8 vs FP32 ONNX
+| Metric            |          FP32 |               INT8 |             Change |
+| ----------------- | ------------: | -----------------: | -----------------: |
+| Model size        |       86.2 MB |        **21.8 MB** | **~74.7% smaller** |
+| CPU latency       |  2.35 ms/text |   **0.77 ms/text** |   **~67.2% lower** |
+| CPU throughput    | 424.7 texts/s | **1299.4 texts/s** | **~205.9% higher** |
+| Session RAM       |       90.4 MB |        **11.2 MB** |   **~87.6% lower** |
+| STS Spearman      |    **0.8138** |             0.8058 |            -0.0080 |
+| SciFact Recall@10 |        0.7342 |         **0.7599** |        **+0.0257** |
+| SciFact nDCG@10   |        0.5897 |         **0.6020** |        **+0.0123** |
+### End-to-End Benchmark Time
+| Model                           | STS Time ↓ | SciFact Time ↓ |
+| ------------------------------- | ---------: | -------------: |
+| **TinyE5-L6-384 (Safetensors)** |      0.72s |          3.71s |
+| TinyE5-L6-384 (FP32 ONNX)       |      8.82s |        265.58s |
+| TinyE5-L6-384 (INT8 ONNX)       |      5.46s |        179.11s |
+| **all-MiniLM-L6-v2**            |  **0.62s** |      **3.62s** |
+> **Summary:** INT8 quantization reduces TinyE5-L6-384 from **86.2 MB to 21.8 MB (~74.7% smaller)**, lowers CPU inference latency from **2.35 ms/text to 0.77 ms/text (~67.2% lower)**, and increases CPU throughput from **424.7 to 1299.4 texts/s (~3.06× throughput)**. Session RAM also drops from **90.4 MB to 11.2 MB (~87.6% lower)**. STS Spearman decreases slightly from **0.8138 to 0.8058**, while SciFact Recall@10 improves from **0.7342 to 0.7599** and nDCG@10 improves from **0.5897 to 0.6020**. The all-MiniLM-L6-v2 baseline still achieves the highest overall quality, with **0.8194 STS Spearman, 0.7923 Recall@10, and 0.6494 nDCG@10**.
 ## Docker Example
       - ./data:/data
     command:
       - --model-id
+      - GrowBitLabs/tinye5
       - --revision
       - int8-onnx
       - --pooling
 All variants are published under the same Hugging Face repository:
 ```text
+GrowBitLabs/tinye5
 ```
 | Variant          | Revision    | Model               |
 ```yaml
 command:
   - --model-id
+  - GrowBitLabs/tinye5
   - --revision
   - main
   - --pooling
 ```yaml
 command:
   - --model-id
+  - GrowBitLabs/tinye5
   - --revision
   - fp32-onnx
   - --pooling
 ```yaml
 command:
   - --model-id
+  - GrowBitLabs/tinye5
   - --revision
   - int8-onnx
   - --pooling

config.json CHANGED Viewed

@@ -1,25 +1,30 @@
-{
-  "architectures": [
-    "BertModel"
-  ],
-  "attention_probs_dropout_prob": 0.1,
-  "classifier_dropout": null,
-  "dtype": "float32",
-  "gradient_checkpointing": false,
-  "hidden_act": "gelu",
-  "hidden_dropout_prob": 0.1,
-  "hidden_size": 384,
-  "initializer_range": 0.02,
-  "intermediate_size": 1536,
-  "layer_norm_eps": 1e-12,
-  "max_position_embeddings": 512,
-  "model_type": "bert",
-  "num_attention_heads": 12,
-  "num_hidden_layers": 6,
-  "pad_token_id": 0,
-  "position_embedding_type": "absolute",
-  "transformers_version": "4.57.6",
-  "type_vocab_size": 2,
-  "use_cache": true,
-  "vocab_size": 30522
-}

+{
+  "add_cross_attention": false,
+  "architectures": [
+    "BertModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": null,
+  "classifier_dropout": null,
+  "dtype": "float32",
+  "eos_token_id": null,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 384,
+  "initializer_range": 0.02,
+  "intermediate_size": 1536,
+  "is_decoder": false,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 6,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "tie_word_embeddings": true,
+  "transformers_version": "5.16.1",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

embedding_config.json CHANGED Viewed

@@ -1,11 +1,11 @@
-{
-  "name": "TinyE5-L6-384",
-  "embedding_dimension": 384,
-  "pooling": "mean",
-  "normalize": true,
-  "query_prefix": "query: ",
-  "passage_prefix": "passage: ",
-  "base_model": "sentence-transformers/all-MiniLM-L6-v2",
-  "training_dataset": "sentence-transformers/msmarco-bm25/triplet",
-  "train_last_n_layers": 2
 }

+{
+  "name": "TinyE5-L6-384",
+  "embedding_dimension": 384,
+  "pooling": "mean",
+  "normalize": true,
+  "query_prefix": "query: ",
+  "passage_prefix": "passage: ",
+  "base_model": "sentence-transformers/all-MiniLM-L6-v2",
+  "training_dataset": "sentence-transformers/msmarco-bm25/triplet",
+  "train_last_n_layers": 2
 }

model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2fb8de9bbf527353997deb64a4a6daee75da86a2eb9e711b825a6e7fa7ed2d01
+size 90864192

tokenizer_config.json CHANGED Viewed

@@ -1,65 +1,24 @@
-{
-  "added_tokens_decoder": {
-    "0": {
-      "content": "[PAD]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "100": {
-      "content": "[UNK]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "101": {
-      "content": "[CLS]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "102": {
-      "content": "[SEP]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "103": {
-      "content": "[MASK]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    }
-  },
-  "clean_up_tokenization_spaces": false,
-  "cls_token": "[CLS]",
-  "do_basic_tokenize": true,
-  "do_lower_case": true,
-  "extra_special_tokens": {},
-  "mask_token": "[MASK]",
-  "max_length": 128,
-  "model_max_length": 512,
-  "never_split": null,
-  "pad_to_multiple_of": null,
-  "pad_token": "[PAD]",
-  "pad_token_type_id": 0,
-  "padding_side": "right",
-  "sep_token": "[SEP]",
-  "stride": 0,
-  "strip_accents": null,
-  "tokenize_chinese_chars": true,
-  "tokenizer_class": "BertTokenizer",
-  "truncation_side": "right",
-  "truncation_strategy": "longest_first",
-  "unk_token": "[UNK]"
-}

+{
+  "backend": "tokenizers",
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "[MASK]",
+  "max_length": 128,
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_to_multiple_of": null,
+  "pad_token": "[PAD]",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}