README.md

metadata

tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - dense
  - generated_from_trainer
  - dataset_size:102127
  - loss:SpladeLoss
  - loss:SparseMultipleNegativesRankingLoss
  - loss:FlopsLoss
base_model: seregadgl/splade_gemma_google_base_checkpoint_100_clear
widget:
  - source_sentence: 'query: 6460338 acdelco'
    sentences:
      - >-
        document: очиститель тормозов rsqprofessional арт  072589767pl
        volkswagen id  buzz янтарный
      - 'document: гтц 6460338 для chevrolet traverse'
      - 'document: гтц 6960358 для chevrolet traverse'
  - source_sentence: 'query: audioquest cinnamon usb 0 7500 см '
    sentences:
      - 'document: кабель usb аудиоквест cinnamon 0 7500 см 8712516'
      - 'document: задняя камера рамке номерного знака интерпауэр ip616 54785862'
      - 'document: аудиокабель soundwave 200 см'
  - source_sentence: 'query: акустическое пианино weber w 121 pw '
    sentences:
      - 'document: акустическое пианино steinway model s'
      - 'document: инструмент для игры на пианино вебер w 121 pw'
      - >-
        document: велосипед сильвербек strela sport 700c 54 см blue
        60097000435025
  - source_sentence: 'query: шкаф шрм24'
    sentences:
      - 'document: wardrobe shrm 24 4348563'
      - 'document: духовой шкаф бертаццони f6011provtn'
      - 'document: шкаф мдф30'
  - source_sentence: 'query: 1452634 santool jawa 300 cl'
    sentences:
      - 'document: смартфон эппл iphone xs max 512gb'
      - 'document: 1453934 santool съемник для сальников jawa 300 cl'
      - 'document: 1452634 santool съемник для сальников jawa 300 cl'
datasets:
  - seregadgl/car_and_product_triplet_103k
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
model-index:
  - name: >-
      SentenceTransformer based on
      seregadgl/splade_gemma_google_base_checkpoint_100_clear
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: val set fine
          type: val_set_fine
        metrics:
          - type: cosine_accuracy@1
            value: 0.742
            name: Cosine Accuracy@1
          - type: cosine_precision@1
            value: 0.742
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.27633333333333326
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.1728
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.08910000000000001
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.742
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.829
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.864
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.891
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.8160719769563038
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.7919432539682544
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.7955622385483846
            name: Cosine Map@100

SentenceTransformer based on seregadgl/splade_gemma_google_base_checkpoint_100_clear

This is a sentence-transformers model finetuned from seregadgl/splade_gemma_google_base_checkpoint_100_clear on the car_and_product_triplet_103k dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: seregadgl/splade_gemma_google_base_checkpoint_100_clear
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • car_and_product_triplet_103k

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): SparseLayer(
    (linear): Linear(in_features=768, out_features=262144, bias=True)
  )
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("seregadgl/splade_gemma_google_base_checkpoint_100_ver2")
# Run inference
sentences = [
    'query: 1452634 santool jawa 300 cl',
    'document: 1452634 santool съемник для сальников jawa 300 cl',
    'document: 1453934 santool съемник для сальников jawa 300 cl',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.1443, 0.1452],
#         [0.1443, 1.0000, 0.7490],
#         [0.1452, 0.7490, 1.0000]])

Evaluation

Metrics

Information Retrieval

• Dataset: val_set_fine
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.742
cosine_precision@1	0.742
cosine_precision@3	0.2763
cosine_precision@5	0.1728
cosine_precision@10	0.0891
cosine_recall@1	0.742
cosine_recall@3	0.829
cosine_recall@5	0.864
cosine_recall@10	0.891
cosine_ndcg@10	0.8161
cosine_mrr@10	0.7919
cosine_map@100	0.7956

Training Details

Training Dataset

car_and_product_triplet_103k

• Dataset: car_and_product_triplet_103k at 3519181
• Size: 102,127 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 5 tokens mean: 16.27 tokens max: 44 tokens	min: 4 tokens mean: 23.62 tokens max: 77 tokens	min: 6 tokens mean: 23.2 tokens max: 47 tokens

• Samples:

  anchor	positive	negative
  query: погружной блендер tefal optichef hb64f810	document: погружной блендер тефаль optichef hb64f810	document: погружной миксер tefal mixchef hb64f850
  query: 375675836 niteo	document: тосол 375675836 для ford f350 полуночный синий	document: тосол 375625836 для ford f350 полуночный синий фиалковый
  query: накидка с подогревом dodge viper pink	document: накидка с подогревом acdelco арт 787327sx dodge viper розовый	document: 787327sx накидка с подогревом indian challenger лаймовый

• Loss: SpladeLoss with these parameters:

  {
      "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score', gather_across_devices=False)",
      "document_regularizer_weight": 1e-05,
      "query_regularizer_weight": 1e-05
  }
  

Evaluation Dataset

car_and_product_triplet_103k

• Dataset: car_and_product_triplet_103k at 3519181
• Size: 1,000 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 5 tokens mean: 16.73 tokens max: 74 tokens	min: 5 tokens mean: 23.54 tokens max: 80 tokens	min: 5 tokens mean: 22.66 tokens max: 65 tokens

• Samples:

  anchor	positive	negative
  query: зеркала для 'слепых' зон volkswagen arteon	document: зеркала для 'слепых' зон 86635985zz для volkswagen arteon перламутровочёрный	document: 86635985zz зеркала для 'слепых' зон иж юпитер2 голубой
  query: elf bar lux 1500 лимонад голубой малины 1500	document: одноразовая электронная сигарета эльф бар 1 5000 мл lemonade blue raspberry 340440526	document: elf bar vibe 1000 мохито зелёного яблока 1000
  query: удалитель наклеек chevrolet corvette onyx	document: удалитель наклеек 20810588pl для chevrolet corvette оникс	document: удалитель наклеек 20810588pl для maserati levante янтарный

• Loss: SpladeLoss with these parameters:

  {
      "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score', gather_across_devices=False)",
      "document_regularizer_weight": 1e-05,
      "query_regularizer_weight": 1e-05
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• gradient_accumulation_steps: 16
• learning_rate: 0.0001
• num_train_epochs: 1
• warmup_steps: 10
• fp16: True
• load_best_model_at_end: True
• router_mapping: {'query': 'anchor', 'document': 'positive'}

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 8
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 16
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 0.0001
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 10
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {'query': 'anchor', 'document': 'positive'}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Validation Loss	val_set_fine_cosine_ndcg@10
0.0125	10	0.8461	0.7841
0.0251	20	0.8195	0.8009
0.0376	30	0.7884	0.7967
0.0501	40	0.7641	0.8097
0.0627	50	0.7503	0.8146
0.0752	60	0.7140	0.8151
0.0877	70	0.7165	0.8180
0.1003	80	0.6955	0.8131
0.1128	90	0.6866	0.8157
0.1253	100	0.6735	0.8170
0.1379	110	0.6766	0.8159
0.1504	120	0.6609	0.8161

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.2.2
• Transformers: 4.57.1
• PyTorch: 2.8.0+cu126
• Accelerate: 1.11.0
• Datasets: 4.4.2
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

SpladeLoss

@misc{formal2022distillationhardnegativesampling,
      title={From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective},
      author={Thibault Formal and Carlos Lassance and Benjamin Piwowarski and Stéphane Clinchant},
      year={2022},
      eprint={2205.04733},
      archivePrefix={arXiv},
      primaryClass={cs.IR},
      url={https://arxiv.org/abs/2205.04733},
}

SparseMultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

FlopsLoss

@article{paria2020minimizing,
    title={Minimizing flops to learn efficient sparse representations},
    author={Paria, Biswajit and Yeh, Chih-Kuan and Yen, Ian EH and Xu, Ning and Ravikumar, Pradeep and P{'o}czos, Barnab{'a}s},
    journal={arXiv preprint arXiv:2004.05665},
    year={2020}
}