davanstrien/iconclass-retriever-bge-ft

Iconclass retriever (bge-base-en-v1.5 fine-tuned)

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5 on the iconclass-retriever-pairs dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-base-en-v1.5
• Maximum Sequence Length: 128 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text
• Training Dataset:
  • iconclass-retriever-pairs
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'cls', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("davanstrien/iconclass-retriever-bge-ft")
# Run inference
queries = [
    'An engraving depicts a dramatic nocturnal scene where a man in period clothing stands on a burning ladder, reaching up to assist a young boy who is climbing down from a high window. The ladder is engulfed in flames at its base, creating a sense of peril, while the boy looks down with a calm expression. In the background, a cityscape is visible under a night sky featuring a radiant sun or moon.',
]
documents = [
    'burning',
    'glowing (heat)',
    'radiating (heat)',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.5269, 0.3512, 0.3070]])

Evaluation

Metrics

Information Retrieval

• Dataset: iconclass
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.2669
cosine_accuracy@5	0.5118
cosine_accuracy@10	0.5929
cosine_precision@1	0.2669
cosine_precision@5	0.1537
cosine_precision@10	0.1054
cosine_precision@24	0.0588
cosine_recall@1	0.0984
cosine_recall@5	0.2548
cosine_recall@10	0.3353
cosine_recall@24	0.4297
cosine_ndcg@10	0.2766
cosine_mrr@10	0.3664
cosine_map@100	0.2174

Training Details

Training Dataset

iconclass-retriever-pairs

• Dataset: iconclass-retriever-pairs
• Size: 21,541 training samples
• Columns: anchor, positive, negative_1, negative_2, negative_3, and negative_4
• Approximate statistics based on the first 100 samples:

  	anchor	positive	negative_1	negative_2	negative_3	negative_4
  type	string	string	string	string	string	string
  modality	text	text	text	text	text	text
  details	min: 58 tokens mean: 81.44 tokens max: 116 tokens	min: 3 tokens mean: 6.9 tokens max: 26 tokens	min: 3 tokens mean: 7.86 tokens max: 27 tokens	min: 3 tokens mean: 8.91 tokens max: 30 tokens	min: 3 tokens mean: 8.61 tokens max: 28 tokens	min: 3 tokens mean: 7.29 tokens max: 24 tokens

• Samples:

  anchor	positive	negative_1	negative_2	negative_3	negative_4
  A dimly lit interior scene depicts a group of horses and a donkey within a rustic stable or ruin. A white horse stands prominently in the center, flanked by a darker horse to its left and another dark horse to its right. In the foreground, a donkey stands near a pile of discarded tack and blankets on the ground, while a small arched window on the left provides a source of light.	barn	farm (building) - AA - civic architecture: inside	saints	imps, familiars of witches and sorcerers	horses (circus performance)
  A vibrant bouquet of various flowers, including roses, tulips, and a hanging crown imperial, is arranged in a dark vase on a wooden ledge. The composition is animated by several butterflies fluttering among the blooms and a small bird's nest with eggs resting on the ledge to the left. A single red tulip lies fallen on the ledge in the foreground, adding a sense of transience to the scene.	flowers	artificial plants and flowers	fabulous plants and herbs	bunch of flowers ~ ornament - AA - stylized	flowers ~ ornament
  A panoramic landscape painting depicts a wide river or lake reflecting the sky, flanked by dense clusters of trees and foliage. In the foreground on the left, a group of travelers walks along a dirt path, while a solitary figure stands nearby. In the background, a town with a prominent church spire and other buildings is visible across the water.	water course	canal	sea (seascape)	landscapes with waters, waterscapes, seascapes (in the temperate zone) - HH - ideal landscapes	heaven represented as paradisaical landscape

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Evaluation Dataset

iconclass-retriever-pairs

• Dataset: iconclass-retriever-pairs
• Size: 200 evaluation samples
• Columns: anchor, positive, negative_1, negative_2, negative_3, and negative_4
• Approximate statistics based on the first 100 samples:

  	anchor	positive	negative_1	negative_2	negative_3	negative_4
  type	string	string	string	string	string	string
  modality	text	text	text	text	text	text
  details	min: 50 tokens mean: 79.51 tokens max: 111 tokens	min: 3 tokens mean: 8.2 tokens max: 33 tokens	min: 3 tokens mean: 8.24 tokens max: 50 tokens	min: 3 tokens mean: 8.41 tokens max: 41 tokens	min: 3 tokens mean: 7.78 tokens max: 25 tokens	min: 3 tokens mean: 7.59 tokens max: 20 tokens

• Samples:

  anchor	positive	negative_1	negative_2	negative_3	negative_4
  A muscular, nude male figure lies supine on a rocky outcrop, his body contorted in a dramatic pose with his head thrown back and his right arm raised toward the sky. A second figure, appearing to be a woman with flowing hair, stands beside him, leaning over and reaching out to touch or support his raised hand. The scene is set against a backdrop of heavy, swirling clouds and a distant landscape, creating a tense and theatrical atmosphere.	David thanks the Lord after he has slain the giant Goliath	David speaks with Goliath	David slings a stone at Goliath's forehead	David beheads Goliath with a sword	lying figure
  A seated female figure, representing the element of water, is depicted in profile facing left. She holds a long, slender vessel from which she pours a stream of water, while a large basket filled with reeds or rushes rests beside her on a rock.	island	dale, valley	coast	forest, wood	plain
  The Virgin Mary, wearing a blue mantle and red dress, sits enthroned holding the Christ Child on her lap. To the left, a Dominican nun and a Dominican friar kneel in prayer, looking up at the holy figures. Above them, a heavenly host of angels and cherubs gathers in a burst of golden light, while two putti at the base of the throne arrange a floral garland.	flowers	artificial plants and flowers	fabulous plants and herbs	standing figure	stand-in

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 64
• learning_rate: 2e-05
• warmup_steps: 0.1
• weight_decay: 0.01
• bf16: True
• per_device_eval_batch_size: 128
• push_to_hub: True
• hub_model_id: davanstrien/iconclass-retriever-bge-ft
• load_best_model_at_end: True
• seed: 12
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• per_device_train_batch_size: 64
• num_train_epochs: 3.0
• max_steps: -1
• learning_rate: 2e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0.1
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.01
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1.0
• label_smoothing_factor: 0.0
• bf16: True
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 128
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: True
• hub_private_repo: None
• hub_model_id: davanstrien/iconclass-retriever-bge-ft
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: True
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 12
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss	iconclass_cosine_ndcg@10
-1	-1	-	-	0.0791
0.0030	1	5.5045	-	-
0.0593	20	5.2547	-	-
0.1187	40	4.9180	-	-
0.1780	60	4.6572	-	-
0.2374	80	4.4856	-	-
0.2967	100	4.3476	-	-
0.3561	120	4.2496	-	-
0.4154	140	4.2427	-	-
0.4748	160	4.1129	-	-
0.5341	180	3.9719	-	-
0.5935	200	4.0231	-	-
0.6528	220	3.9212	-	-
0.7122	240	3.9147	-	-
0.7507	253	-	1.7970	0.2702
0.7715	260	3.9227	-	-
0.8309	280	3.8546	-	-
0.8902	300	3.3277	-	-
0.9496	320	1.9490	-	-
1.0089	340	2.1030	-	-
1.0682	360	3.6301	-	-
1.1276	380	3.5009	-	-
1.1869	400	3.6163	-	-
1.2463	420	3.5507	-	-
1.3056	440	3.5883	-	-
1.3650	460	3.5163	-	-
1.4243	480	3.5287	-	-
1.4837	500	3.4211	-	-
1.5015	506	-	1.6725	0.2785
1.5430	520	3.5846	-	-
1.6024	540	3.4772	-	-
1.6617	560	3.4992	-	-
1.7211	580	3.4459	-	-
1.7804	600	3.4747	-	-
1.8398	620	3.4815	-	-
1.8991	640	2.7377	-	-
1.9585	660	1.7949	-	-
2.0178	680	2.0796	-	-
2.0772	700	3.3525	-	-
2.1365	720	3.3296	-	-
2.1958	740	3.2522	-	-
2.2522	759	-	1.5502	0.2842
2.2552	760	3.2960	-	-
2.3145	780	3.2314	-	-
2.3739	800	3.2461	-	-
2.4332	820	3.2065	-	-
2.4926	840	3.1930	-	-
2.5519	860	3.2079	-	-
2.6113	880	3.2920	-	-
2.6706	900	3.2403	-	-
2.7300	920	3.2088	-	-
2.7893	940	3.3953	-	-
2.8487	960	3.2497	-	-
2.9080	980	2.5122	-	-
2.9674	1000	1.7967	-	-
3.0	1011	-	1.5537	0.2766

Training Time

• Training: 8.1 minutes
• Evaluation: 18.6 seconds
• Total: 8.4 minutes

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.5.1
• Transformers: 5.10.2
• PyTorch: 2.12.0+cu130
• Accelerate: 1.13.0
• Datasets: 4.8.5
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}