OloriBern/musique-mixer-2000

CrossEncoder based on BAAI/bge-reranker-v2-m3

This is a Cross Encoder model finetuned from BAAI/bge-reranker-v2-m3 using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Model Details

Model Description

• Model Type: Cross Encoder
• Base model: BAAI/bge-reranker-v2-m3
• Maximum Sequence Length: 1024 tokens
• Number of Output Labels: 1 label

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("cross_encoder_model_id")
# Get scores for pairs of texts
pairs = [
    ['Who played Mr. Tucker of the show by the person who plays the old man in Waiting on A Woman?', "</s></s>Waitin' on a Woman. Paisley has referred to ``Waitin 'on a Woman ''as`` one of the most important songs'' that he's ever recorded. Because of the importance that he places on the song, Paisley asked Andy Griffith to star in the music video, as he felt that Griffith's personality matched the personality of the older man in the song. Griffith speaks the old man's lines in the video as well. Jim Shea and Peter Tilden directed the video.</s></s></s>"],
    ['What culture\'s arrival in the country the performer of Privilege is a citizen of is known as the "Davidian Revolution"?', '</s></s></s>Ivor Cutler. In 2014 a new play, "The Beautiful Cosmos of Ivor Cutler", a co-production by Vanishing Point and National Theatre of Scotland, was performed.</s></s>'],
    ['What position was held by the Republican candidate running for governor in the state Plum Hollow Country Club is located?', '</s></s>Gerald Hills. Gerald (Rusty) J. Hills, II is an American politician and educator in the state of Michigan and is currently the communications director for Michigan Attorney General Bill Schuette.</s></s></s>2018 Michigan gubernatorial election. Michigan gubernatorial election, 2018 ← 2014 November 6, 2018 2022 → Nominee Bill Schuette Gretchen Whitmer Bill Gelineau Party Republican Democratic Libertarian Running mate Lisa Posthumus Lyons Garlin Gilchrist II Angelique Thomas Incumbent Governor Rick Snyder Republican'],
    ['When did weed become legal where marble for the lincoln memorial was sourced?', '</s>Cannabis in California. Cannabis in California is permitted, subject to regulations, for both medical and recreational use. In recent decades the state has led the country in efforts to legalize cannabis, holding the first (unsuccessful) vote to decriminalize it in 1972 and, through Proposition 215, becoming the first state to legalize it for medical use in 1996. In the November 2016 election, voters passed an amendment legalizing recreational use of marijuana.</s></s></s>Timeline of cannabis laws in the United States. The legal history of cannabis in the United States began with state - level prohibition in the early 20th century, with the first major federal limitations occurring in 1937. Starting with Oregon in 1973, individual states began to liberalize cannabis laws through decriminalization. In 1996, California became the first state to legalize medical cannabis, sparking a trend that spread to a majority of states by 2016. In 2012, Colorado and Washington became the first states to legalize cannabis for recreational use.</s>'],
    ['Where is the Rio Grande river located in the country where Norbrook is located?', 'Norfolk Island. Norfolk Island is located in the South Pacific Ocean, east of the Australian mainland. Norfolk Island is the main island of the island group the territory encompasses and is located at 29°02′S 167°57′E\ufeff / \ufeff29.033°S 167.950°E\ufeff / -29.033; 167.950. It has an area of 34.6 square kilometres (13.4 sq mi), with no large-scale internal bodies of water and 32 km (20 mi) of coastline. The island\'s highest point is Mount Bates (319 metres (1,047 feet) above sea level), located in the northwest quadrant of the island. The majority of the terrain is suitable for farming and other agricultural uses. Phillip Island, the second largest island of the territory, is located at 29°07′S 167°57′E\ufeff / \ufeff29.117°S 167.950°E\ufeff / -29.117; 167.950, seven kilometres (4.3 miles) south of the main island.</s>Norfolk Island. Norfolk Island is located in the South Pacific Ocean, east of the Australian mainland. Norfolk Island is the main island of the island group the territory encompasses and is located at 29°02′S 167°57′E\ufeff / \ufeff29.033°S 167.950°E\ufeff / -29.033; 167.950. It has an area of 34.6 square kilometres (13.4 sq mi), with no large-scale internal bodies of water and 32 km (20 mi) of coastline. The island\'s highest point is Mount Bates (319 metres (1,047 feet) above sea level), located in the northwest quadrant of the island. The majority of the terrain is suitable for farming and other agricultural uses. Phillip Island, the second largest island of the territory, is located at 29°07′S 167°57′E\ufeff / \ufeff29.117°S 167.950°E\ufeff / -29.117; 167.950, seven kilometres (4.3 miles) south of the main island.</s></s>Norbrook. Norbrook is an upscale neighbourhood of the Kingston Metropolitan Area of Jamaica, with approximately 15,000 residents and is an important residential, shopping and commercial centre of the city itself. Norbrook is regarded as anywhere from the Immaculate Conception High School (in the South) to about 100m up "The Hill" (in the North).</s></s>Rio Grande (Jamaica). The Rio Grande is a river of Jamaica, found in the parish of Portland. It was named when the Spanish occupied Jamaica in the 15th and 16th centuries. One of the largest rivers in Jamaica, it was named ``Big River \'\'(Rio Grande) by the Spanish, and today is one of the many tourist attractions in Portland, mainly for rafting.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    'Who played Mr. Tucker of the show by the person who plays the old man in Waiting on A Woman?',
    [
        "</s></s>Waitin' on a Woman. Paisley has referred to ``Waitin 'on a Woman ''as`` one of the most important songs'' that he's ever recorded. Because of the importance that he places on the song, Paisley asked Andy Griffith to star in the music video, as he felt that Griffith's personality matched the personality of the older man in the song. Griffith speaks the old man's lines in the video as well. Jim Shea and Peter Tilden directed the video.</s></s></s>",
        '</s></s></s>Ivor Cutler. In 2014 a new play, "The Beautiful Cosmos of Ivor Cutler", a co-production by Vanishing Point and National Theatre of Scotland, was performed.</s></s>',
        '</s></s>Gerald Hills. Gerald (Rusty) J. Hills, II is an American politician and educator in the state of Michigan and is currently the communications director for Michigan Attorney General Bill Schuette.</s></s></s>2018 Michigan gubernatorial election. Michigan gubernatorial election, 2018 ← 2014 November 6, 2018 2022 → Nominee Bill Schuette Gretchen Whitmer Bill Gelineau Party Republican Democratic Libertarian Running mate Lisa Posthumus Lyons Garlin Gilchrist II Angelique Thomas Incumbent Governor Rick Snyder Republican',
        '</s>Cannabis in California. Cannabis in California is permitted, subject to regulations, for both medical and recreational use. In recent decades the state has led the country in efforts to legalize cannabis, holding the first (unsuccessful) vote to decriminalize it in 1972 and, through Proposition 215, becoming the first state to legalize it for medical use in 1996. In the November 2016 election, voters passed an amendment legalizing recreational use of marijuana.</s></s></s>Timeline of cannabis laws in the United States. The legal history of cannabis in the United States began with state - level prohibition in the early 20th century, with the first major federal limitations occurring in 1937. Starting with Oregon in 1973, individual states began to liberalize cannabis laws through decriminalization. In 1996, California became the first state to legalize medical cannabis, sparking a trend that spread to a majority of states by 2016. In 2012, Colorado and Washington became the first states to legalize cannabis for recreational use.</s>',
        'Norfolk Island. Norfolk Island is located in the South Pacific Ocean, east of the Australian mainland. Norfolk Island is the main island of the island group the territory encompasses and is located at 29°02′S 167°57′E\ufeff / \ufeff29.033°S 167.950°E\ufeff / -29.033; 167.950. It has an area of 34.6 square kilometres (13.4 sq mi), with no large-scale internal bodies of water and 32 km (20 mi) of coastline. The island\'s highest point is Mount Bates (319 metres (1,047 feet) above sea level), located in the northwest quadrant of the island. The majority of the terrain is suitable for farming and other agricultural uses. Phillip Island, the second largest island of the territory, is located at 29°07′S 167°57′E\ufeff / \ufeff29.117°S 167.950°E\ufeff / -29.117; 167.950, seven kilometres (4.3 miles) south of the main island.</s>Norfolk Island. Norfolk Island is located in the South Pacific Ocean, east of the Australian mainland. Norfolk Island is the main island of the island group the territory encompasses and is located at 29°02′S 167°57′E\ufeff / \ufeff29.033°S 167.950°E\ufeff / -29.033; 167.950. It has an area of 34.6 square kilometres (13.4 sq mi), with no large-scale internal bodies of water and 32 km (20 mi) of coastline. The island\'s highest point is Mount Bates (319 metres (1,047 feet) above sea level), located in the northwest quadrant of the island. The majority of the terrain is suitable for farming and other agricultural uses. Phillip Island, the second largest island of the territory, is located at 29°07′S 167°57′E\ufeff / \ufeff29.117°S 167.950°E\ufeff / -29.117; 167.950, seven kilometres (4.3 miles) south of the main island.</s></s>Norbrook. Norbrook is an upscale neighbourhood of the Kingston Metropolitan Area of Jamaica, with approximately 15,000 residents and is an important residential, shopping and commercial centre of the city itself. Norbrook is regarded as anywhere from the Immaculate Conception High School (in the South) to about 100m up "The Hill" (in the North).</s></s>Rio Grande (Jamaica). The Rio Grande is a river of Jamaica, found in the parish of Portland. It was named when the Spanish occupied Jamaica in the 15th and 16th centuries. One of the largest rivers in Jamaica, it was named ``Big River \'\'(Rio Grande) by the Spanish, and today is one of the many tourist attractions in Portland, mainly for rafting.',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Evaluation

Metrics

Cross Encoder Correlation

• Datasets: validation and train_subset
• Evaluated with CECorrelationEvaluator

Metric	validation	train_subset
pearson	0.9038	0.8992
spearman	0.8998	0.896

Training Details

Training Dataset

Unnamed Dataset

• Size: 20,049 training samples
• Columns: sentence_0, sentence_1, and label
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1	label
  type	string	string	float
  details	min: 37 characters mean: 95.53 characters max: 251 characters	min: 20 characters mean: 1038.52 characters max: 4881 characters	min: 0.0 mean: 0.5 max: 1.0

• Samples:

  sentence_0	sentence_1	label
  Who played Mr. Tucker of the show by the person who plays the old man in Waiting on A Woman?	Waitin' on a Woman. Paisley has referred to Waitin 'on a Woman ''as one of the most important songs'' that he's ever recorded. Because of the importance that he places on the song, Paisley asked Andy Griffith to star in the music video, as he felt that Griffith's personality matched the personality of the older man in the song. Griffith speaks the old man's lines in the video as well. Jim Shea and Peter Tilden directed the video.	0.5
  What culture's arrival in the country the performer of Privilege is a citizen of is known as the "Davidian Revolution"?	Ivor Cutler. In 2014 a new play, "The Beautiful Cosmos of Ivor Cutler", a co-production by Vanishing Point and National Theatre of Scotland, was performed.	0.3333333333333333
  What position was held by the Republican candidate running for governor in the state Plum Hollow Country Club is located?	Gerald Hills. Gerald (Rusty) J. Hills, II is an American politician and educator in the state of Michigan and is currently the communications director for Michigan Attorney General Bill Schuette.2018 Michigan gubernatorial election. Michigan gubernatorial election, 2018 ← 2014 November 6, 2018 2022 → Nominee Bill Schuette Gretchen Whitmer Bill Gelineau Party Republican Democratic Libertarian Running mate Lisa Posthumus Lyons Garlin Gilchrist II Angelique Thomas Incumbent Governor Rick Snyder Republican	0.6666666666666666

• Loss: BinaryCrossEntropyLoss with these parameters:

  {
      "activation_fn": "torch.nn.modules.linear.Identity",
      "pos_weight": null
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 2
• per_device_eval_batch_size: 2

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 2
• per_device_eval_batch_size: 2
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• eval_use_gather_object: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	validation_spearman	train_subset_spearman
0.0499	250	-	0.8300	0.8134
0.0997	500	0.517	0.8536	0.8442
0.1496	750	-	0.8496	0.8538
0.1995	1000	0.4395	0.8586	0.8600
0.2494	1250	-	0.8661	0.8649
0.2992	1500	0.4236	0.8770	0.8673
0.3491	1750	-	0.8788	0.8715
0.3990	2000	0.4354	0.8829	0.8765
0.4488	2250	-	0.8810	0.8766
0.4987	2500	0.4056	0.8835	0.8819
0.5486	2750	-	0.8857	0.8828
0.5984	3000	0.4093	0.8858	0.8842
0.6483	3250	-	0.8940	0.8858
0.6982	3500	0.4207	0.8905	0.8893
0.7481	3750	-	0.8954	0.8937
0.7979	4000	0.4006	0.8960	0.8942
0.8478	4250	-	0.8998	0.8960

Framework Versions

• Python: 3.11.13
• Sentence Transformers: 5.2.2
• Transformers: 4.44.2
• PyTorch: 2.10.0+cu128
• Accelerate: 1.12.0
• Datasets: 4.0.0
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}