Upload v2.0: Improved document context support

README.md

@@ -5,20 +5,35 @@ tags:
 - feature-extraction
 - dense
 - generated_from_trainer
-- dataset_size:10
+- dataset_size:32
 - loss:CosineSimilarityLoss
 base_model: sentence-transformers/all-MiniLM-L6-v2
 widget:
-- source_sentence: cloudinit config
+- source_sentence: cloud init
   sentences:
-  - user data bootstrap
-  - user-data yaml
-  - userdata script
+  - EC2 instance user data
+  - CFT parameters
+  - user data
+- source_sentence: cloud-init
+  sentences:
+  - user data configuration
+  - Setting up user data for EC2
+  - Parameters
+- source_sentence: user data
+  sentences:
+  - user data guide
+  - Cloud-init configuration guide
+  - network security
+- source_sentence: cloud-init
+  sentences:
+  - Using cloud-init for bootstrapping
+  - user data configuration
+  - CREATE_FAILED error in CloudFormation stack
 - source_sentence: cloud-init script
   sentences:
-  - network
-  - userdata
+  - Cloud-init setup
   - user data script
+  - initialization script
 pipeline_tag: sentence-similarity
 library_name: sentence-transformers
 ---
@@ -75,7 +90,7 @@ model = SentenceTransformer("yagosys/cloudinit-embedding")
 sentences = [
     'cloud-init script',
     'user data script',
-    'network',
+    'initialization script',
 ]
 embeddings = model.encode(sentences)
 print(embeddings.shape)
@@ -84,9 +99,9 @@ print(embeddings.shape)
 # Get the similarity scores for the embeddings
 similarities = model.similarity(embeddings, embeddings)
 print(similarities)
-# tensor([[ 1.0000,  0.9679, -0.0970],
-#         [ 0.9679,  1.0000, -0.1298],
-#         [-0.0970, -0.1298,  1.0000]])
+# tensor([[1.0000, 0.9762, 0.7631],
+#         [0.9762, 1.0000, 0.7589],
+#         [0.7631, 0.7589, 1.0000]])
 ```
 
 <!--
@@ -131,19 +146,19 @@ You can finetune this model on your own dataset.
 
 #### Unnamed Dataset
 
-* Size: 10 training samples
+* Size: 32 training samples
 * Columns: <code>sentence_0</code>, <code>sentence_1</code>, and <code>label</code>
-* Approximate statistics based on the first 10 samples:
-  |         | sentence_0                                                                     | sentence_1                                                                     | label                                                         |
-  |:--------|:-------------------------------------------------------------------------------|:-------------------------------------------------------------------------------|:--------------------------------------------------------------|
-  | type    | string                                                                         | string                                                                         | float                                                         |
-  | details | <ul><li>min: 4 tokens</li><li>mean: 6.5 tokens</li><li>max: 9 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 5.0 tokens</li><li>max: 7 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.8</li><li>max: 1.0</li></ul> |
+* Approximate statistics based on the first 32 samples:
+  |         | sentence_0                                                                      | sentence_1                                                                       | label                                                          |
+  |:--------|:--------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|:---------------------------------------------------------------|
+  | type    | string                                                                          | string                                                                           | float                                                          |
+  | details | <ul><li>min: 4 tokens</li><li>mean: 5.53 tokens</li><li>max: 9 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 6.56 tokens</li><li>max: 18 tokens</li></ul> | <ul><li>min: 0.1</li><li>mean: 0.71</li><li>max: 1.0</li></ul> |
 * Samples:
-  | sentence_0                        | sentence_1                       | label            |
-  |:----------------------------------|:---------------------------------|:-----------------|
-  | <code>cloud-init bootstrap</code> | <code>user data bootstrap</code> | <code>1.0</code> |
-  | <code>cloudinit</code>            | <code>user-data</code>           | <code>1.0</code> |
-  | <code>user data</code>            | <code>network</code>             | <code>0.0</code> |
+  | sentence_0              | sentence_1                                      | label            |
+  |:------------------------|:------------------------------------------------|:-----------------|
+  | <code>cloud-init</code> | <code>EC2 launch</code>                         | <code>0.5</code> |
+  | <code>user data</code>  | <code>Using cloud-init for bootstrapping</code> | <code>0.9</code> |
+  | <code>cloud-init</code> | <code>Parameters</code>                         | <code>0.2</code> |
 * Loss: [<code>CosineSimilarityLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosinesimilarityloss) with these parameters:
   ```json
   {
@@ -154,9 +169,7 @@ You can finetune this model on your own dataset.
 ### Training Hyperparameters
 #### Non-Default Hyperparameters
 
-- `per_device_train_batch_size`: 2
-- `per_device_eval_batch_size`: 2
-- `num_train_epochs`: 20
+- `num_train_epochs`: 30
 - `multi_dataset_batch_sampler`: round_robin
 
 #### All Hyperparameters
@@ -166,8 +179,8 @@ You can finetune this model on your own dataset.
 - `do_predict`: False
 - `eval_strategy`: no
 - `prediction_loss_only`: True
-- `per_device_train_batch_size`: 2
-- `per_device_eval_batch_size`: 2
+- `per_device_train_batch_size`: 8
+- `per_device_eval_batch_size`: 8
 - `per_gpu_train_batch_size`: None
 - `per_gpu_eval_batch_size`: None
 - `gradient_accumulation_steps`: 1
@@ -179,7 +192,7 @@ You can finetune this model on your own dataset.
 - `adam_beta2`: 0.999
 - `adam_epsilon`: 1e-08
 - `max_grad_norm`: 1
-- `num_train_epochs`: 20
+- `num_train_epochs`: 30
 - `max_steps`: -1
 - `lr_scheduler_type`: linear
 - `lr_scheduler_kwargs`: {}

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6b404e433583e4fa007c7ab91c0257bd818221a9a0decb678e15088675d39ab3
+oid sha256:f13147cd7edef87394ef4d8f7f8b203651cca52a567577316ba6d64b993eb209
 size 90864192