Hugging Face's logo

Tayyab112
/
crossencoder-hardneg

Text Ranking
sentence-transformers
Safetensors
bert
cross-encoder
reranker
Generated from Trainer
dataset_size:20000
loss:BinaryCrossEntropyLoss
dataset_size:15447
text-embeddings-inference
Model card Files
xet
Community
crossencoder-hardneg / README.md
Tayyab112's picture
Tayyab112
Upload folder using huggingface_hub
d4df3ab verified
preview code
|
raw
history blame contribute delete
16 kB
 ---
tags:
- sentence-transformers
- cross-encoder
- reranker
- generated_from_trainer
- dataset_size:20000
- loss:BinaryCrossEntropyLoss
- dataset_size:15447
pipeline_tag: text-ranking
library_name: sentence-transformers
---
# CrossEncoder
This is a [Cross Encoder](https://www.sbert.net/docs/cross_encoder/usage/usage.html) model trained using the [sentence-transformers](https://www.SBERT.net) library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.
## Model Details
### Model Description
- **Model Type:** Cross Encoder
<!-- - **Base model:** [Unknown](https://huggingface.co/unknown) -->
- **Maximum Sequence Length:** 128 tokens
- **Number of Output Labels:** 1 label
<!-- - **Training Dataset:** Unknown -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Documentation:** [Cross Encoder Documentation](https://www.sbert.net/docs/cross_encoder/usage/usage.html)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers)
- **Hugging Face:** [Cross Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=cross-encoder)
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import CrossEncoder
# Download from the 🤗 Hub
model = CrossEncoder("cross_encoder_model_id")
# Get scores for pairs of texts
pairs = [
['where is edmonton', "Edmonton Map â\x80\x94 Satellite Images of Edmonton. detailed map of Edmonton and near places. Welcome to the Edmonton google satellite map! This place is situated in Division No. 11, Alberta, Canada, its geographical coordinates are 53° 33' 0 North, 113° 30' 0 West and its original name (with diacritics) is Edmonton."],
['where is monkey gum trail', 'Getting to Monkey Gum Fire Trail from the Turpentine Road is not to easy and will require good map reading skills and a bit of luck for those trying it for the first time. It is located on a disused part of the road and as can be seen in this image is the sign, once you manage to find it, is not that great.onkey Gum Fire Trail â\x80\x93 Death Adder: These are not a snake to be playing with. One bite from this guy and you wonâ\x80\x99t be home for dinner. These are fairly common in the area so keep an eye out where you tread.'],
['where do void salts come from?', 'The Purified Void Salts is a quest item found in The Elder Scrolls V: Dawnguard. It is a quest item for the quest Chasing Echoes. It can be found in the old laboratory of Valerica. It lies in the large bowl next to the fire salts and normal void salts on a shelf on the top floor of the chamber.'],
['where is american canyon ca', "Overview of American Canyon. Located about 10 miles south of Napa, American Canyon, Calif., is a lovely and green city in the heart of California's wine country. Luckily for urbanites, the city is also just 35 miles northeast of exciting San Francisco."],
['where is mount rushmore', "Mount Rushmore. History & Culture. Mount Rushmore, also known as the President's Mountain, is located in the Black Hills of Keystone, South Dakota. The sculpture of four famous presidents, George Washington, Thomas Jefferson, Theodore Roosevelt, and Abraham Lincoln, was carved into the granite rock face."],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)
# Or rank different texts based on similarity to a single text
ranks = model.rank(
'where is edmonton',
[
"Edmonton Map â\x80\x94 Satellite Images of Edmonton. detailed map of Edmonton and near places. Welcome to the Edmonton google satellite map! This place is situated in Division No. 11, Alberta, Canada, its geographical coordinates are 53° 33' 0 North, 113° 30' 0 West and its original name (with diacritics) is Edmonton.",
'Getting to Monkey Gum Fire Trail from the Turpentine Road is not to easy and will require good map reading skills and a bit of luck for those trying it for the first time. It is located on a disused part of the road and as can be seen in this image is the sign, once you manage to find it, is not that great.onkey Gum Fire Trail â\x80\x93 Death Adder: These are not a snake to be playing with. One bite from this guy and you wonâ\x80\x99t be home for dinner. These are fairly common in the area so keep an eye out where you tread.',
'The Purified Void Salts is a quest item found in The Elder Scrolls V: Dawnguard. It is a quest item for the quest Chasing Echoes. It can be found in the old laboratory of Valerica. It lies in the large bowl next to the fire salts and normal void salts on a shelf on the top floor of the chamber.',
"Overview of American Canyon. Located about 10 miles south of Napa, American Canyon, Calif., is a lovely and green city in the heart of California's wine country. Luckily for urbanites, the city is also just 35 miles northeast of exciting San Francisco.",
"Mount Rushmore. History & Culture. Mount Rushmore, also known as the President's Mountain, is located in the Black Hills of Keystone, South Dakota. The sculpture of four famous presidents, George Washington, Thomas Jefferson, Theodore Roosevelt, and Abraham Lincoln, was carved into the granite rock face.",
]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
```
<!--
### Direct Usage (Transformers)
<details><summary>Click to see the direct usage in Transformers</summary>
</details>
-->
<!--
### Downstream Usage (Sentence Transformers)
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
</details>
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
<!--
## Bias, Risks and Limitations
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->
<!--
### Recommendations
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->
## Training Details
### Training Dataset
#### Unnamed Dataset
* Size: 15,447 training samples
* Columns: <code>sentence_0</code>, <code>sentence_1</code>, and <code>label</code>
* Approximate statistics based on the first 1000 samples:
| | sentence_0 | sentence_1 | label |
|:--------|:-----------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------|:--------------------------------------------------------------|
| type | string | string | float |
| details | <ul><li>min: 12 characters</li><li>mean: 29.65 characters</li><li>max: 85 characters</li></ul> | <ul><li>min: 91 characters</li><li>mean: 338.32 characters</li><li>max: 976 characters</li></ul> | <ul><li>min: 1.0</li><li>mean: 1.0</li><li>max: 1.0</li></ul> |
* Samples:
| sentence_0 | sentence_1 | label |
|:--------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------|
| <code>where is edmonton</code> | <code>Edmonton Map — Satellite Images of Edmonton. detailed map of Edmonton and near places. Welcome to the Edmonton google satellite map! This place is situated in Division No. 11, Alberta, Canada, its geographical coordinates are 53° 33' 0 North, 113° 30' 0 West and its original name (with diacritics) is Edmonton.</code> | <code>1.0</code> |
| <code>where is monkey gum trail</code> | <code>Getting to Monkey Gum Fire Trail from the Turpentine Road is not to easy and will require good map reading skills and a bit of luck for those trying it for the first time. It is located on a disused part of the road and as can be seen in this image is the sign, once you manage to find it, is not that great.onkey Gum Fire Trail – Death Adder: These are not a snake to be playing with. One bite from this guy and you won’t be home for dinner. These are fairly common in the area so keep an eye out where you tread.</code> | <code>1.0</code> |
| <code>where do void salts come from?</code> | <code>The Purified Void Salts is a quest item found in The Elder Scrolls V: Dawnguard. It is a quest item for the quest Chasing Echoes. It can be found in the old laboratory of Valerica. It lies in the large bowl next to the fire salts and normal void salts on a shelf on the top floor of the chamber.</code> | <code>1.0</code> |
* Loss: [<code>BinaryCrossEntropyLoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#binarycrossentropyloss) with these parameters:
```json
{
"activation_fn": "torch.nn.modules.linear.Identity",
"pos_weight": null
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 32
- `num_train_epochs`: 1
#### All Hyperparameters
<details><summary>Click to expand</summary>
- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: no
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 32
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 5e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1
- `num_train_epochs`: 1
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: None
- `warmup_ratio`: 0.0
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `bf16`: False
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `parallelism_config`: None
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch_fused
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `project`: huggingface
- `trackio_space_id`: trackio
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `hub_revision`: None
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`:
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: no
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `liger_kernel_config`: None
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: True
- `prompts`: None
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: proportional
- `router_mapping`: {}
- `learning_rate_mapping`: {}
</details>
### Training Logs
| Epoch | Step | Training Loss |
|:-----:|:----:|:-------------:|
| 0.8 | 500 | 0.0 |
### Framework Versions
- Python: 3.12.12
- Sentence Transformers: 5.2.0
- Transformers: 4.57.6
- PyTorch: 2.9.0+cu126
- Accelerate: 1.12.0
- Datasets: 4.0.0
- Tokenizers: 0.22.2
## Citation
### BibTeX
#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
```
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->