Tayyab112/crossencoder-hardneg

CrossEncoder

This is a Cross Encoder model trained using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Model Details

Model Description

• Model Type: Cross Encoder
• Maximum Sequence Length: 128 tokens
• Number of Output Labels: 1 label

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("cross_encoder_model_id")
# Get scores for pairs of texts
pairs = [
    ['where is edmonton', "Edmonton Map â\x80\x94 Satellite Images of Edmonton. detailed map of Edmonton and near places. Welcome to the Edmonton google satellite map! This place is situated in Division No. 11, Alberta, Canada, its geographical coordinates are 53° 33' 0 North, 113° 30' 0 West and its original name (with diacritics) is Edmonton."],
    ['where is monkey gum trail', 'Getting to Monkey Gum Fire Trail from the Turpentine Road is not to easy and will require good map reading skills and a bit of luck for those trying it for the first time. It is located on a disused part of the road and as can be seen in this image is the sign, once you manage to find it, is not that great.onkey Gum Fire Trail â\x80\x93 Death Adder: These are not a snake to be playing with. One bite from this guy and you wonâ\x80\x99t be home for dinner. These are fairly common in the area so keep an eye out where you tread.'],
    ['where do void salts come from?', 'The Purified Void Salts is a quest item found in The Elder Scrolls V: Dawnguard. It is a quest item for the quest Chasing Echoes. It can be found in the old laboratory of Valerica. It lies in the large bowl next to the fire salts and normal void salts on a shelf on the top floor of the chamber.'],
    ['where is american canyon ca', "Overview of American Canyon. Located about 10 miles south of Napa, American Canyon, Calif., is a lovely and green city in the heart of California's wine country. Luckily for urbanites, the city is also just 35 miles northeast of exciting San Francisco."],
    ['where is mount rushmore', "Mount Rushmore. History & Culture. Mount Rushmore, also known as the President's Mountain, is located in the Black Hills of Keystone, South Dakota. The sculpture of four famous presidents, George Washington, Thomas Jefferson, Theodore Roosevelt, and Abraham Lincoln, was carved into the granite rock face."],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    'where is edmonton',
    [
        "Edmonton Map â\x80\x94 Satellite Images of Edmonton. detailed map of Edmonton and near places. Welcome to the Edmonton google satellite map! This place is situated in Division No. 11, Alberta, Canada, its geographical coordinates are 53° 33' 0 North, 113° 30' 0 West and its original name (with diacritics) is Edmonton.",
        'Getting to Monkey Gum Fire Trail from the Turpentine Road is not to easy and will require good map reading skills and a bit of luck for those trying it for the first time. It is located on a disused part of the road and as can be seen in this image is the sign, once you manage to find it, is not that great.onkey Gum Fire Trail â\x80\x93 Death Adder: These are not a snake to be playing with. One bite from this guy and you wonâ\x80\x99t be home for dinner. These are fairly common in the area so keep an eye out where you tread.',
        'The Purified Void Salts is a quest item found in The Elder Scrolls V: Dawnguard. It is a quest item for the quest Chasing Echoes. It can be found in the old laboratory of Valerica. It lies in the large bowl next to the fire salts and normal void salts on a shelf on the top floor of the chamber.',
        "Overview of American Canyon. Located about 10 miles south of Napa, American Canyon, Calif., is a lovely and green city in the heart of California's wine country. Luckily for urbanites, the city is also just 35 miles northeast of exciting San Francisco.",
        "Mount Rushmore. History & Culture. Mount Rushmore, also known as the President's Mountain, is located in the Black Hills of Keystone, South Dakota. The sculpture of four famous presidents, George Washington, Thomas Jefferson, Theodore Roosevelt, and Abraham Lincoln, was carved into the granite rock face.",
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Training Details

Training Dataset

Unnamed Dataset

• Size: 15,447 training samples
• Columns: sentence_0, sentence_1, and label
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1	label
  type	string	string	float
  details	min: 12 characters mean: 29.65 characters max: 85 characters	min: 91 characters mean: 338.32 characters max: 976 characters	min: 1.0 mean: 1.0 max: 1.0

• Samples:

  sentence_0	sentence_1	label
  where is edmonton	Edmonton Map — Satellite Images of Edmonton. detailed map of Edmonton and near places. Welcome to the Edmonton google satellite map! This place is situated in Division No. 11, Alberta, Canada, its geographical coordinates are 53° 33' 0 North, 113° 30' 0 West and its original name (with diacritics) is Edmonton.	1.0
  where is monkey gum trail	Getting to Monkey Gum Fire Trail from the Turpentine Road is not to easy and will require good map reading skills and a bit of luck for those trying it for the first time. It is located on a disused part of the road and as can be seen in this image is the sign, once you manage to find it, is not that great.onkey Gum Fire Trail – Death Adder: These are not a snake to be playing with. One bite from this guy and you won’t be home for dinner. These are fairly common in the area so keep an eye out where you tread.	1.0
  where do void salts come from?	The Purified Void Salts is a quest item found in The Elder Scrolls V: Dawnguard. It is a quest item for the quest Chasing Echoes. It can be found in the old laboratory of Valerica. It lies in the large bowl next to the fire salts and normal void salts on a shelf on the top floor of the chamber.	1.0

• Loss: BinaryCrossEntropyLoss with these parameters:

  {
      "activation_fn": "torch.nn.modules.linear.Identity",
      "pos_weight": null
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• num_train_epochs: 1

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.8	500	0.0

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.2.0
• Transformers: 4.57.6
• PyTorch: 2.9.0+cu126
• Accelerate: 1.12.0
• Datasets: 4.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}