File size: 24,979 Bytes
f269399 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 |
---
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- dense
- generated_from_trainer
- dataset_size:637
- loss:TripletLoss
base_model: sentence-transformers/all-MiniLM-L6-v2
widget:
- source_sentence: "We are the data controller in respect of your personal data and\
\ will handle your data in accordance with \nour obligations under the Privacy\
\ Laws. We will use this information solely in connection with \nadministering\
\ the Championship and exploiting the rights granted to us pursuant to any separate\
\ \nagreement entered into with your team or otherwise. We are entitled to do\
\ so on the basis of our \nlegitimate interests, namely to enable us to operate\
\ the Championship and promote and exploit your \nparticipation in the same."
sentences:
- The aerodynamic design of the new F1 car's rear wing has been optimized to reduce
drag and improve downforce, allowing drivers to reach higher speeds on the straights.
- As the data controller, we will manage your personal information in accordance
with privacy laws, using it solely to administer the Formula 1 Championship and
promote your participation.
- The engine's ability to produce power is directly related to the pressure of the
fuel-air mixture it receives. As the pressure increases, so does the potential
for power output, with atmospheric pressure serving as the maximum threshold for
normally aspirated engines.
- source_sentence: "With adjustments \nfor temperature, altitude, and other factors,\
\ the EPR gauge \npresents an indication of the thrust being developed by \nthe\
\ engine. Since the EPR gauge compares two pressures, \nit is a differential pressure\
\ gauge. It is a remote-sensing \ninstrument that receives its input from an engine\
\ pressure \nratio transmitter or, in digital instrument systems displays, \n\
from a computer."
sentences:
- The engine's fuel efficiency can be significantly improved by implementing advanced
materials in the engine's components.
- Prior to the start of the race, all personnel except drivers, officials, and technical
staff must vacate the grid within 10 minutes. At the 5-minute signal, all cars
on the grid and in the pit lane must have their wheels fitted, and tyre blankets
disconnected from power supply. Team personnel and equipment trolleys must begin
leaving the grid at this time.
- The EPR gauge on an engine provides a reading of the thrust being produced by
adjusting for various factors such as temperature and altitude, and it does this
by comparing two different pressures, making it a type of differential pressure
gauge that receives its input from a remote-sensing instrument.
- source_sentence: "57.5 Unless asked to do so by the FIA, cars may not be moved from\
\ the fast lane whilst the sprint \nsession or the race is suspended. Any driver\
\ whose car is moved from the fast lane to any other \npart of the pit lane will\
\ be arranged at the back of the line of cars in the fast lane in the order \n\
they got there. At all times drivers must follow the directions of the marshals."
sentences:
- The aerodynamic design of F1 cars has led to significant advancements in downforce,
allowing drivers to take corners at higher speeds and improving overall racing
performance.
- A detailed record of repair work, including photographs and part numbers, must
be maintained. Additionally, gears, dog rings, and reverse components can be changed
under supervision during a competition if they are damaged, but significant parts
of a car's RNC cannot be replaced between competitions without FIA permission.
- During a suspended sprint session or race, F1 cars are not to be moved from the
fast lane unless instructed by the FIA. If a driver's car is moved to a different
part of the pit lane, they will be placed at the back of the fast lane in the
order they arrived. Marshals' instructions must be followed at all times.
- source_sentence: "If such conditions are not met, then the \nPower Unit Manufacturer\
\ may, at its sole and exclusive discretion, decline the request to supply \n\
such New Customer Team and the decline of such request shall not be deemed to\
\ be a breach \nof the terms set out in this Appendix (however Article c) cannot\
\ be applied or interpreted by the \nPower Unit Manufacturer in a way which would\
\ d eprive the obligation of supply as referred to \nin Article b) above of any\
\ effect and/or that would prevent the FIA from making and enforcing \nthe provisions\
\ set out in Article b) above. The Power Unit Manufacturer undertakes to exercise\
\ \nin good faith the co nditions referred to in paragraph 1 to 11 below). The\
\ teams and the Power \nUnit Manufacturers remain free to negotiate the terms\
\ of the supply agreement, subject to the \nfall-back positions set out below\
\ which shall apply should a team and a Power Unit Manufacturer \nfail to reach\
\ an agreement, despite negotiating in good faith."
sentences:
- The clerk of the course has the authority to temporarily halt practice sessions
to ensure the track is clear or to assist in the recovery of a vehicle. During
qualifying or sprint qualifying sessions, the session duration may be extended
as a result of interruptions. Any disputes regarding the impact of these interruptions
on driver qualification will not be accepted.
- If a new customer team fails to meet the required conditions, the power unit manufacturer
has the right to decline their supply request. However, this decision cannot be
used to circumvent the obligation to supply as stated in the agreement. Both parties
must negotiate in good faith, and if they fail to reach an agreement, the terms
set out below will apply.
- The aerodynamic design of the new generation of F1 cars has led to a significant
increase in downforce, but at the cost of reduced fuel efficiency and increased
tire wear.
- source_sentence: "10.7 Information to be provided to the FIA and Competitors \n\
a) In order that an FIA observer may be appointed, Competitors must inform the\
\ FIA and all \nother Competitors of any planned TPC, PE or DE at least 72 hours\
\ before it is due to \ncommence, and the following information must be provided:\
\ \ni) The precise specification of the car(s) to be used. ii) The name(s) of\
\ the driver(s). iii) The type of activity."
sentences:
- Competitors must notify the FIA and other teams at least 72 hours in advance of
any planned technical testing, physical evaluations, or development exercises,
providing detailed information about the cars, drivers, and nature of the activity.
- Pitot tubes can be either covered or uncovered, and modifications to the Driver
Cooling Scoop, as outlined in Article 3.6.5 of the Technical Regulations, are
also permitted. Additionally, changes can be made to improve the driver's comfort.
- The aerodynamic design of a Formula 1 car's rear wing is crucial in determining
its overall downforce and drag characteristics, requiring a delicate balance between
speed and stability.
datasets:
- zacCMU/RAG_FINETUNING_For_Engineering
pipeline_tag: sentence-similarity
library_name: sentence-transformers
---
# SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) on the [rag_finetuning_for_engineering](https://huggingface.co/datasets/zacCMU/RAG_FINETUNING_For_Engineering) dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
## Model Details
### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) <!-- at revision c9745ed1d9f207416be6d2e6f8de32d1f16199bf -->
- **Maximum Sequence Length:** 256 tokens
- **Output Dimensionality:** 384 dimensions
- **Similarity Function:** Cosine Similarity
- **Training Dataset:**
- [rag_finetuning_for_engineering](https://huggingface.co/datasets/zacCMU/RAG_FINETUNING_For_Engineering)
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
### Full Model Architecture
```
SentenceTransformer(
(0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'BertModel'})
(1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("zacCMU/miniLM2-ENG")
# Run inference
sentences = [
'10.7 Information to be provided to the FIA and Competitors \na) In order that an FIA observer may be appointed, Competitors must inform the FIA and all \nother Competitors of any planned TPC, PE or DE at least 72 hours before it is due to \ncommence, and the following information must be provided: \ni) The precise specification of the car(s) to be used. ii) The name(s) of the driver(s). iii) The type of activity.',
'Competitors must notify the FIA and other teams at least 72 hours in advance of any planned technical testing, physical evaluations, or development exercises, providing detailed information about the cars, drivers, and nature of the activity.',
"The aerodynamic design of a Formula 1 car's rear wing is crucial in determining its overall downforce and drag characteristics, requiring a delicate balance between speed and stability.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000, -0.6238, 0.9745],
# [-0.6238, 1.0000, -0.6029],
# [ 0.9745, -0.6029, 1.0000]])
```
<!--
### Direct Usage (Transformers)
<details><summary>Click to see the direct usage in Transformers</summary>
</details>
-->
<!--
### Downstream Usage (Sentence Transformers)
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
</details>
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
<!--
## Bias, Risks and Limitations
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->
<!--
### Recommendations
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->
## Training Details
### Training Dataset
#### rag_finetuning_for_engineering
* Dataset: [rag_finetuning_for_engineering](https://huggingface.co/datasets/zacCMU/RAG_FINETUNING_For_Engineering) at [bddb325](https://huggingface.co/datasets/zacCMU/RAG_FINETUNING_For_Engineering/tree/bddb32566fad4c8fea0d7ea081f1eaf8b4192dd7)
* Size: 637 training samples
* Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
* Approximate statistics based on the first 637 samples:
| | anchor | positive | negative |
|:--------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
| type | string | string | string |
| details | <ul><li>min: 9 tokens</li><li>mean: 112.88 tokens</li><li>max: 256 tokens</li></ul> | <ul><li>min: 18 tokens</li><li>mean: 60.28 tokens</li><li>max: 176 tokens</li></ul> | <ul><li>min: 15 tokens</li><li>mean: 35.03 tokens</li><li>max: 57 tokens</li></ul> |
* Samples:
| anchor | positive | negative |
|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| <code>A penalty in accordance with Article 54.3d) will be imposed on any <br>driver who fails to start the race from the pit lane. If any driver needs assistance after the fifteen (15) second signal, he must raise his arm and, <br>when the remainder of the cars able to do so have left the pit lane, marshals will be instructed <br>to push the car into the inner lane. In this case, marshals with yellow flags will stand beside any <br>car concerned to warn drivers behind.</code> | <code>A driver who fails to start the race from the pit lane will incur a penalty. If a driver requires assistance after the 15-second signal, they must signal for help and marshals will then guide their car into the inner lane, warning other drivers with yellow flags.</code> | <code>The aerodynamic design of modern Formula 1 cars requires a delicate balance between downforce and drag to achieve optimal speed on the track.</code> |
| <code>If a driver wishes to leave his car before it is weighed , he <br>must ask the Technical Delegate to weigh him in order that this weight may be added to <br>that of the car. e) If a car stops on the circuit during the qualifying session or the sprint qualifying session <br>and the driver leaves the car, he must go to the FIA garage immediately on his return to <br>the pit lane in order for his weight to be established. 35.2 After the sprint session or the race any classified car may be weighed.</code> | <code>To avoid penalties, a driver must ensure their weight is accurately recorded before leaving their car, either by having the Technical Delegate weigh them or by being weighed in the FIA garage after returning to the pit lane. This process is crucial during qualifying sessions, sprint qualifying sessions, and after the sprint session or the race.</code> | <code>The aerodynamic design of a Formula 1 car's rear wing plays a crucial role in generating downforce, but its impact on the overall handling and stability of the vehicle is often overlooked by teams in their pursuit of speed.</code> |
| <code>d) When leaving the pits a driver may overtake, or be overtaken by, another car on the track <br>before he reaches the second safety car line. e) When the safety car is returning to the pits it may be overtaken by cars on the track once <br>it has reached the first safety car line. f) Whilst in the pit entry road, pit lane or pit exit road a driver may overtake another car <br>which is also in one of these three areas.</code> | <code>When exiting the pits, a driver is allowed to overtake or be overtaken by another car on the track before reaching the second safety car line. Additionally, the safety car can be overtaken by cars on the track once it has reached the first safety car line, and drivers can also overtake each other while in the pit entry road, pit lane, or pit exit road.</code> | <code>The aerodynamic design of modern Formula 1 cars relies heavily on complex computational fluid dynamics simulations to optimize their downforce and drag characteristics.</code> |
* Loss: [<code>TripletLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#tripletloss) with these parameters:
```json
{
"distance_metric": "TripletDistanceMetric.EUCLIDEAN",
"triplet_margin": 5
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `per_device_train_batch_size`: 16
- `learning_rate`: 1e-05
- `num_train_epochs`: 4
#### All Hyperparameters
<details><summary>Click to expand</summary>
- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: no
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 8
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 1e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 4
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.0
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `bf16`: False
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `parallelism_config`: None
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch_fused
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `project`: huggingface
- `trackio_space_id`: trackio
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `hub_revision`: None
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`:
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: no
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `liger_kernel_config`: None
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: True
- `prompts`: None
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: proportional
- `router_mapping`: {}
- `learning_rate_mapping`: {}
</details>
### Training Logs
| Epoch | Step | Training Loss |
|:-----:|:----:|:-------------:|
| 0.25 | 10 | 5.5019 |
| 0.5 | 20 | 5.2724 |
| 0.75 | 30 | 5.1275 |
| 1.0 | 40 | 4.999 |
| 1.25 | 50 | 4.8488 |
| 1.5 | 60 | 4.7919 |
| 1.75 | 70 | 4.6734 |
| 2.0 | 80 | 4.4696 |
| 2.25 | 90 | 4.4078 |
| 2.5 | 100 | 4.2232 |
| 2.75 | 110 | 4.1736 |
| 3.0 | 120 | 4.0837 |
| 3.25 | 130 | 4.0113 |
| 3.5 | 140 | 4.0376 |
| 3.75 | 150 | 3.9134 |
| 4.0 | 160 | 3.9853 |
### Framework Versions
- Python: 3.12.12
- Sentence Transformers: 5.1.2
- Transformers: 4.57.1
- PyTorch: 2.9.0+cu126
- Accelerate: 1.11.0
- Datasets: 4.0.0
- Tokenizers: 0.22.1
## Citation
### BibTeX
#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
```
#### TripletLoss
```bibtex
@misc{hermans2017defense,
title={In Defense of the Triplet Loss for Person Re-Identification},
author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
year={2017},
eprint={1703.07737},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
--> |