KerasHub
laxmareddyp commited on
Commit
bf0442b
·
verified ·
1 Parent(s): 2d0856a

Update README.md with new model card content

Browse files
Files changed (1) hide show
  1. README.md +105 -17
README.md CHANGED
@@ -1,20 +1,108 @@
1
  ---
2
  library_name: keras-hub
3
  ---
4
- This is a [`Bert` model](https://keras.io/api/keras_hub/models/bert) uploaded using the KerasHub library and can be used with JAX, TensorFlow, and PyTorch backends.
5
- This model is related to a `TextEmbedder` task.
6
-
7
- Model config:
8
- * **name:** bert_backbone
9
- * **trainable:** True
10
- * **dtype:** {'module': 'keras', 'class_name': 'DTypePolicy', 'config': {'name': 'float32'}, 'registered_name': None}
11
- * **vocabulary_size:** 250037
12
- * **num_layers:** 12
13
- * **num_heads:** 12
14
- * **hidden_dim:** 384
15
- * **intermediate_dim:** 1536
16
- * **dropout:** 0.1
17
- * **max_sequence_length:** 512
18
- * **num_segments:** 2
19
-
20
- This model card has been generated automatically and should be completed by the model author. See [Model Cards documentation](https://huggingface.co/docs/hub/model-cards) for more information.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  library_name: keras-hub
3
  ---
4
+ ### Model Overview
5
+ ## Model Summary
6
+
7
+ Multilingual E5 is a family of multilingual text embedding models from [intfloat ](https://huggingface.co/intfloat), based on the XLM-RoBERTa architecture. These models generate fixed-size sentence embeddings suitable for semantic search, retrieval, clustering, and text classification across 100+ languages.
8
+
9
+ The E5 models were introduced in ["Multilingual E5 Text Embeddings: A Technical Report"](https://arxiv.org/abs/2402.05672) by Wang et al. The models are trained in two stages: (1) contrastive pre-training on ~1 billion multilingual text pairs, and (2) fine-tuning on labeled datasets. Input text should be prefixed with "query: " or "passage: " for optimal performance.
10
+
11
+ ## Key Features:
12
+ - Supports 100+ languages via XLM-RoBERTa backbone
13
+ - Trained on ~1 billion multilingual text pairs
14
+ - MIT licensed
15
+ - Works out of the box for semantic search, retrieval, clustering, and classification
16
+ - Model Family: Multilingual E5
17
+ - Architecture: XLM-RoBERTa (bidirectional Transformer encoder)
18
+ - Languages: 100+
19
+ - Task Type: Text Embedding / Sentence Similarity / Retrieval
20
+ - Max Sequence Length: 512 tokens
21
+
22
+ ## Model Details
23
+
24
+ * [Multilingual E5 Quickstart Notebook](coming soon..)
25
+ * [Multilingual E5 API Documentation](https://keras.io/keras_hub/api/models/xlm_roberta/)
26
+ * [Multilingual E5 Model Card](https://huggingface.co/intfloat/multilingual-e5-small)
27
+ * [Multilingual E5 Technical Paper](https://arxiv.org/abs/2402.05672)
28
+ * [KerasHub Beginner Guide](https://keras.io/guides/keras_hub/getting_started/)
29
+ * [KerasHub Model Publishing Guide](https://keras.io/guides/keras_hub/upload/)
30
+
31
+ ## Installation
32
+
33
+ Keras and KerasHub can be installed with:
34
+
35
+ ```
36
+ pip install -U -q keras-hub
37
+ pip install -U -q keras
38
+
39
+ ```
40
+
41
+ Jax, TensorFlow, and Torch come preinstalled in Kaggle Notebooks. For instructions on installing them in another environment see the [Keras Getting Started](https://keras.io/getting_started/) page.
42
+
43
+ ## Presets
44
+
45
+ ## Preset Table
46
+
47
+ | Preset | Architecture | Pooling | Normalize | Languages | Description |
48
+ |---|---|---|---|---|---|
49
+ | `multilingual_e5_small` | XLM-RoBERTa | Mean | L2 | 100+ | 12-layer Multilingual E5 small model. Produces 384-dim embeddings. |
50
+ | `multilingual_e5_base` | XLM-RoBERTa | Mean | L2 | 100+ | 12-layer Multilingual E5 base model. Produces 768-dim embeddings. |
51
+ | `multilingual_e5_large` | XLM-RoBERTa | Mean | L2 | 100+ | 24-layer Multilingual E5 large model. Produces 1024-dim embeddings. |
52
+
53
+ ## Example Usage
54
+ ```
55
+ import keras_hub
56
+
57
+ # Load the text embedder with preprocessing.
58
+ embedder = keras_hub.models.TextEmbedder.from_preset(
59
+ "multilingual_e5_small"
60
+ )
61
+
62
+ # Encode queries
63
+ query = "query: Which planet is known as the Red Planet?"
64
+ q_emb = embedder.encode_text(query)
65
+
66
+ # Encode documents/passages
67
+ documents = [
68
+ "passage: Mars is often referred to as the Red Planet.",
69
+ "passage: Venus is often called Earth's twin.",
70
+ ]
71
+ d_embs = embedder.encode_text(documents)
72
+ ```
73
+
74
+ ```
75
+ # Load just the backbone for custom architectures.
76
+ backbone = keras_hub.models.XLMRobertaBackbone.from_preset(
77
+ "multilingual_e5_small",
78
+ )
79
+ ```
80
+
81
+ ## Example Usage with Hugging Face URI
82
+
83
+ ```
84
+ import keras_hub
85
+
86
+ # Load the text embedder with preprocessing.
87
+ embedder = keras_hub.models.TextEmbedder.from_preset(
88
+ "hf://keras/multilingual_e5_small"
89
+ )
90
+
91
+ # Encode queries
92
+ query = "query: Which planet is known as the Red Planet?"
93
+ q_emb = embedder.encode_text(query)
94
+
95
+ # Encode documents/passages
96
+ documents = [
97
+ "passage: Mars is often referred to as the Red Planet.",
98
+ "passage: Venus is often called Earth's twin.",
99
+ ]
100
+ d_embs = embedder.encode_text(documents)
101
+ ```
102
+
103
+ ```
104
+ # Load just the backbone for custom architectures.
105
+ backbone = keras_hub.models.XLMRobertaBackbone.from_preset(
106
+ "hf://keras/multilingual_e5_small",
107
+ )
108
+ ```