KerasHub
laxmareddyp commited on
Commit
69ccdab
·
verified ·
1 Parent(s): 3551359

Update README.md with new model card content

Browse files
Files changed (1) hide show
  1. README.md +157 -16
README.md CHANGED
@@ -1,19 +1,160 @@
1
  ---
2
  library_name: keras-hub
3
  ---
4
- This is a [`XLMRoberta` model](https://keras.io/api/keras_hub/models/xlm_roberta) uploaded using the KerasHub library and can be used with JAX, TensorFlow, and PyTorch backends.
5
- This model is related to a `TextEmbedder` task.
6
-
7
- Model config:
8
- * **name:** xlm_roberta_backbone
9
- * **trainable:** True
10
- * **dtype:** {'module': 'keras', 'class_name': 'DTypePolicy', 'config': {'name': 'float32'}, 'registered_name': None}
11
- * **vocabulary_size:** 250002
12
- * **num_layers:** 24
13
- * **num_heads:** 16
14
- * **hidden_dim:** 1024
15
- * **intermediate_dim:** 4096
16
- * **dropout:** 0.1
17
- * **max_sequence_length:** 8192
18
-
19
- This model card has been generated automatically and should be completed by the model author. See [Model Cards documentation](https://huggingface.co/docs/hub/model-cards) for more information.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  library_name: keras-hub
3
  ---
4
+ ### Model Overview
5
+ # BGE
6
+
7
+ BGE (BAAI General Embedding) models for dense text retrieval and semantic similarity tasks, implemented in Keras.
8
+
9
+ ## Model Overview
10
+
11
+ BGE (BAAI General Embedding) is a family of bi-directional, transformer-based text embedding models developed by the Beijing Academy of Artificial Intelligence (BAAI). Built on the `BERT `encoder architecture, BGE models are fine-tuned specifically for dense retrieval, semantic similarity, and clustering tasks.
12
+
13
+ For embedding generation, the model outputs `L2-normalized` embeddings of the [`CLS] token's` hidden state, producing fixed-dimensional dense vectors suitable for cosine similarity comparisons.
14
+
15
+ These models can be used with KerasHub through the `BgeTextEmbedder` task API.
16
+
17
+ ## Architecture
18
+
19
+ BGE models follow the standard BERT encoder architecture:
20
+
21
+ * Tokenizer: WordPiece tokenizer with BERT-compatible special tokens ([CLS], [SEP], [PAD]).
22
+
23
+ * Encoder: Multi-layer bi-directional Transformer encoder.
24
+
25
+ * Embedding output: L2-normalized [CLS] token hidden states.
26
+
27
+ ## Intended Use
28
+
29
+ * Semantic search and information retrieval
30
+ * Document similarity and clustering
31
+ * Retrieval-Augmented Generation (RAG) pipelines
32
+ * Question-answer matching
33
+
34
+ ## Training Data
35
+
36
+ BGE models are trained on large-scale text pair datasets for contrastive learning. See the [original paper ](https://arxiv.org/pdf/2309.07597)and [BAAI's Hugging Face page](https://huggingface.co/BAAI) for full training details.
37
+ ## Links
38
+
39
+ * [BGE Quickstart Notebook](coming soon..)
40
+ * [BGE API Documentation](https://keras.io/keras_hub/api/models/bert/)
41
+ * [BGE Model Card](https://huggingface.co/BAAI/bge-small-en)
42
+ * [Original Paper](https://arxiv.org/pdf/2309.07597)
43
+ * [BGE](https://huggingface.co/BAAI)
44
+ * [KerasHub Beginner Guide](https://keras.io/guides/keras_hub/getting_started/)
45
+ * [KerasHub Model Publishing Guide](https://keras.io/guides/keras_hub/upload/)
46
+
47
+ ## Installation
48
+
49
+ Keras and KerasHub can be installed with:
50
+
51
+ ```
52
+ pip install -U -q keras-hub
53
+ pip install -U -q keras
54
+
55
+ ```
56
+
57
+ Jax, TensorFlow, and Torch come preinstalled in Kaggle Notebooks. For instructions on installing them in another environment see the [Keras Getting Started](https://keras.io/getting_started/) page.
58
+
59
+ ## Presets
60
+
61
+ | Preset | Architecture | Pooling | Normalize | Languages |
62
+ |---|---|---|---|---|
63
+ | `bge_small_en` | BERT | CLS | L2 | English |
64
+ | `bge_base_en` | BERT | CLS | L2 | English |
65
+ | `bge_large_en` | BERT | CLS | L2 | English |
66
+ | `bge_small_v1.5_en` | BERT | CLS | L2 | English |
67
+ | `bge_base_v1.5_en` | BERT | CLS | L2 | English |
68
+ | `bge_large_v1.5_en` | BERT | CLS | L2 | English |
69
+ | `bge_base_zh` | BERT | CLS | L2 | Chinese |
70
+ | `bge_large_zh` | BERT | CLS | L2 | Chinese |
71
+ | `bge_small_v1.5_zh` | BERT | CLS | L2 | Chinese |
72
+ | `bge_base_v1.5_zh` | BERT | CLS | L2 | Chinese |
73
+ | `bge_large_v1.5_zh` | BERT | CLS | L2 | Chinese |
74
+ | `bge_llm_embedder` | BERT | CLS | L2 | English |
75
+ | `bge_m3` | XLM-RoBERTa | CLS | L2 | 100+ |
76
+
77
+ ## Example Usage
78
+ ```
79
+
80
+ # Install and setup
81
+ !pip install -q keras-hub
82
+
83
+ import os
84
+ os.environ["KERAS_BACKEND"] = "jax" # or "tensorflow" or "torch"
85
+
86
+ import keras_hub
87
+ import numpy as np
88
+
89
+ # Load a BGE model from the Kaggle preset
90
+ embedder = keras_hub.models.BertTextEmbedder.from_preset("bge_m3")
91
+
92
+ # Encode text into embeddings
93
+ embeddings = embedder.encode_text(["The weather is lovely today."])
94
+ print(f"Shape: {embeddings.shape}") # (1, 384)
95
+
96
+ # Compute similarity between sentences
97
+ query = ["What is deep learning?"]
98
+ passages = [
99
+ "Deep learning is a subset of machine learning using neural networks with many layers.",
100
+ "The Eiffel Tower is located in Paris, France.",
101
+ "Neural networks learn representations of data through backpropagation.",
102
+ ]
103
+
104
+ # Encode the texts into embeddings before passing to similarity
105
+ query_embeddings = embedder.encode_text(query)
106
+ passage_embeddings = embedder.encode_documents(passages)
107
+
108
+ # Calculate similarity
109
+ scores = embedder.similarity(query_embeddings, passage_embeddings)
110
+
111
+ print("Similarity scores:")
112
+ # Access the first row of scores [0] since we have 1 query
113
+ for passage, score in zip(passages, np.array(scores)[0]):
114
+ print(f" {float(score):.4f} → {passage[:60]}...")
115
+
116
+
117
+ ```
118
+
119
+ ## Example Usage with Hugging Face URI
120
+
121
+ ```
122
+
123
+ # Install and setup
124
+ !pip install -q keras-hub
125
+
126
+ import os
127
+ os.environ["KERAS_BACKEND"] = "jax" # or "tensorflow" or "torch"
128
+
129
+ import keras_hub
130
+ import numpy as np
131
+
132
+ # Load a BGE model from the Kaggle preset
133
+ embedder = keras_hub.models.BertTextEmbedder.from_preset("hf://keras/bge_m3")
134
+
135
+ # Encode text into embeddings
136
+ embeddings = embedder.encode_text(["The weather is lovely today."])
137
+ print(f"Shape: {embeddings.shape}") # (1, 384)
138
+
139
+ # Compute similarity between sentences
140
+ query = ["What is deep learning?"]
141
+ passages = [
142
+ "Deep learning is a subset of machine learning using neural networks with many layers.",
143
+ "The Eiffel Tower is located in Paris, France.",
144
+ "Neural networks learn representations of data through backpropagation.",
145
+ ]
146
+
147
+ # Encode the texts into embeddings before passing to similarity
148
+ query_embeddings = embedder.encode_text(query)
149
+ passage_embeddings = embedder.encode_documents(passages)
150
+
151
+ # Calculate similarity
152
+ scores = embedder.similarity(query_embeddings, passage_embeddings)
153
+
154
+ print("Similarity scores:")
155
+ # Access the first row of scores [0] since we have 1 query
156
+ for passage, score in zip(passages, np.array(scores)[0]):
157
+ print(f" {float(score):.4f} → {passage[:60]}...")
158
+
159
+
160
+ ```