mariakrissmer/alias_demo_model

SentenceTransformer based on NeuML/pubmedbert-base-embeddings

This is a sentence-transformers model finetuned from NeuML/pubmedbert-base-embeddings. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: NeuML/pubmedbert-base-embeddings
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("mariakrissmer/alias_demo_model")
# Run inference
sentences = [
    'FTL, FTH1, TMSB4X, B2M, MALAT1, LYZ, ACTB, TMSB10, RHOG, S100A11, TYROBP, S100A6, CST3, EEF1A1, PFN1, AIF1, CFL1, TPT1, CD52, S100A4, SH3BGRL3, HLA-DRA, UBA52, LILRB2, FCER1G, CD74, FCN1, PSAP, CYBA, PTMA, DUSP1, GNB2L1, SAT1, COTL1, OAZ1, VIM, H3F3B, HLA-DPA1, MYL6, SRSF5, NPC2, ZFP36, HLA-B, NACA, EIF1, ACTG1, LGALS1, CTSS, ARPC2, HLA-E expression pattern defines this as a CD14+ Monocytes cell.',
    'The expression of FTL, FTH1, S100A9, TMSB4X, TMSB10, S100A4, B2M, S100A8, LYZ, EEF1A1, ACTB, FOS, CTSS, EIF1, OAZ1, S100A11, S100A6, TKT, CD74, GNB2L1, MALAT1, TPT1, CYBA, JUNB, TYROBP, TYMP, FCER1G, PTMA, HLA-C, LST1, PPDPF, IER2, SH3BGRL3, LAPTM5, TXNIP, ID2, GPX1, GPSM3, LAMTOR4, SAT1, KLF6, VIM, PSAP, GAPDH, ARHGDIB, ALDOA, PFN1, ASGR1, CD68, CST3 aligns with a CD14+ Monocytes identity.',
    'A transcriptome with MALAT1, B2M, TMSB4X, RARS, TPT1, ARPC5, EEF1B2, EEF1A1, H3F3B, MYL12B, PTMA, UBE2D3, ACTB, STK17A, GIMAP7, HNRNPK, CFL1, ARHGDIB, EIF1, PSMA7, UBA52, AL592183.1, TRAF3IP3, SRSF7, CALM2, WIPF1, UBE2E3, CNBP, LAP3, FAM175A, H2AFZ, OSTC, CCDC109B, COX7C, DUSP1, HLA-E, HLA-C, PIM1, CYCS, PPIA, SLC25A6, RBM3, EEF1D, PDCD4-AS1, FAU, ATP5L, PFDN5, HNRNPA1, BTG1, CALM1 points toward CD4 T cells identity.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.9911, 0.3080],
#         [0.9911, 1.0000, 0.3525],
#         [0.3080, 0.3525, 1.0000]])

Evaluation

Metrics

Triplet

• Dataset: triplet_eval_scrna
• Evaluated with TripletEvaluator

Metric	Value
cosine_accuracy	0.929

Training Details

Training Dataset

Unnamed Dataset

• Size: 10,676 training samples
• Columns: sentence1, sentence2, and negative
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	negative
  type	string	string	string
  details	min: 164 tokens mean: 181.83 tokens max: 204 tokens	min: 164 tokens mean: 181.5 tokens max: 204 tokens	min: 164 tokens mean: 181.71 tokens max: 204 tokens

• Samples:

  sentence1	sentence2	negative
  Expression of FTL, TMSB4X, FTH1, B2M, ACTB, MALAT1, CCT7, COTL1, SPG7, OAZ1, RAB4B, CST3, AIF1, S100A4, PFN1, LGALS1, LST1, EEF1A1, CYBA, TMSB10, FAU, NACA, GNB2L1, HLA-DRA, HLA-DPA1, VIM, S100A6, PTMA, TKT, IFITM3, LYZ, SAT1, SH3BGRL3, FCER1G, TIMP1, LAS1L, ARPC3, TYROBP, HLA-C, FCN1, IFITM2, GAPDH, TPT1, CD52, YBX1, FCGR3A, CD74, PABPC1, STXBP2, HLA-B suggests FCGR3A+ Monocytes lineage commitment.	The transcriptome suggests a FCGR3A+ Monocytes type, with expression of B2M, FTH1, TMSB4X, FTL, ACTB, MALAT1, JUNB, TNFSF10, LAMTOR4, HDAC5, LYZ, OAZ1, TYROBP, CTSS, FCGR3A, AIF1, FCER1G, TMSB10, HLA-C, IFITM2, SAT1, CST3, EEF1A1, NACA, PFN1, CD74, GNB2L1, HLA-DPA1, NCKAP1L, COTL1, EIF1, ARPC2, LST1, ARHGDIB, SH3BGRL3, PTMA, SERPINA1, CYBA, ACTG1, EMP3, CD52, NCF2, HLA-A, CHCHD2, ARPC1B, EEF1D, PFDN5, HNRNPA1, ARPC3, ALDOA.	CD4 T cells cells are known to express: MALAT1, TMSB4X, B2M, EEF1A1, JUN, JUNB, HLA-A, TMSB10, ACTB, TPT1, FAU, CXCR4, EEF1D, H3F3B, ZFP36L2, TMA7, HLA-C, HNRNPA1, PFN1, EIF1, FTL, TXNIP, DUSP1, GNB2L1, ARHGDIB, PFDN5, FOS, SRP14, MYL12A, EEF2, EIF3K, ZFP36, CD52, LAPTM5, S100A4, CD48, ARPC2, ARL6IP5, COX7C, HNRNPA0, HLA-B, LTB, ANXA1, ATP6V1G1, VIM, LDHB, MYL6, BTG1, ARL6IP4, IL32.
  With genes like ACTB, TMSB4X, B2M, GAPDH, PTMA, ABRACL, TMSB10, PPP1CA, ACTG1, CFL1, THOC7, RARRES3, TUBA1B, PFN1, EEF1A1, H2AFZ, HNRNPA2B1, HMGB1, HNRNPA1, RAN, NPM1, PPIA, CORO1A, SRRM1, LDHA, TPI1, NACA, HSP90AA1, EIF4A1, ENO1, TUBB, CHCHD2, FTH1, ARHGDIB, MYL6, COTL1, EIF4A3, YBX1, HMGB2, VIM, FAU, MALAT1, ATP5G2, CALM1, COX4I1, FTL, ACTR3, CD74, GNB2L1, HLA-C active, this cell is identified as a CD8 T cells.	Observed top genes: MALAT1, TMSB4X, B2M, JUNB, PGK1, ACTB, LTB, TPT1, EIF1, MX2, S100A4, PTMA, ACTG1, S100A6, EEF1A1, MYL12A, UBA52, NPM1, HLA-C, CALM1, CD52, TXNIP, ID2, DUSP1, GNB2L1, HLA-A, CD99, VIM, FAU, CFL1, HSPA8, NACA, SLC25A3, FOS, IL32, PFN1, EIF3K, GLTSCR2, FTL, CD2, HCLS1, PLAC8, GZMK, GRK6, HLA-F, CITED2, ARPC1B, IL2RG, HNRNPK, CASP4.	A cell that expresses the following genes: FTL, FTH1, TMSB4X, LYZ, S100A4, MALAT1, S100A9, PTMA, TMSB10, VIM, PFN1, FAU, GAPDH, TPT1, S100A8, S100A6, B2M, EIF1, LGALS2, LGALS1, CTSS, S100A10, S100A11, CD74, DUSP1, NACA, CST3, OAZ1, TYROBP, SH3BGRL3, EEF1B2, GNB2L1, HLA-B, LST1, AIF1, EEF1A1, ACTB, FOS, H3F3B, UBA52, ISG15, LAPTM5, FCER1G, RAC1, NCF1, PABPC1, KLF6, CFL1, PFDN5, MT2A is likely a CD14+ Monocytes cell.
  This cell likely originates from the CD4 T cells family, based on expression of MALAT1, B2M, TMSB4X, TPT1, EEF1A1, FAU, TMSB10, JUNB, PTMA, HLA-C, EEF1D, GNB2L1, ABRACL, ACTB, PABPC1, DUSP1, FTH1, NACA, HLA-E, BTG1, HLA-B, TOMM7, CFL1, FOS, PFN1, EIF1, DDX5, SH3BGRL3, ARPC5, EEF1B2, PLAC8, ANAPC16, LSP1, HNRNPA1, HMGB1, IL32, COX4I1, CCR7, LIMD2, FTL, RBBP4, CXCR4, CCNI, COX7C, HLA-A, LTB, SRSF3, NDUFA1, SPOCK2, EIF3F.	MALAT1, ALDOA, TMSB4X, B2M, EEF1A1, ACTB, TPT1, TMSB10, FTH1, HLA-C, C6orf48, FOS, UBA52, TAGLN2, PTMA, GNB2L1, BTG1, EIF1, H3F3B, FTL, CD52, CXCR4, TMEM66, VIM, NACA, CIRBP, JUN, S100A10, EEF1B2, NPM1, HLA-A, UCP2, TMEM123, LDHB, HNRNPA1, GLTSCR2, SUN2, SH3BGRL3, ZCCHC11, DUSP1, HLA-E, HLA-B, EIF3F, CD44, ARHGDIB, ARPC3, SRP14, IL32, UBB, DDX5 define the expression landscape of this cell.	MALAT1, B2M, ACTB, TMSB4X, SUV420H2, HLA-C, NKG7, COMMD10, CCL5, HLA-A, HLA-B, TMSB10, GZMB, PFN1, TXNIP, GNLY, RFNG, ARPC2, MSN, FKBP11, COMMD6, UBB, EIF1, H3F3B, ACTG1, PPDPF, FTL, RAC2, S100A4, FCGR3A, FGFBP2, GZMA, HLA-E, CLIC1, EEF1D, TYROBP, CNBP, SPON2, AGA, BTF3, NDUFA4, GIMAP7, IFITM2, HNRNPA1, MYL6, UBC, TPT1, SERF2, GPATCH8, CD7 reflect the unique expression profile of NK cells cells.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 1,187 evaluation samples
• Columns: sentence1, sentence2, and negative
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	negative
  type	string	string	string
  details	min: 164 tokens mean: 181.73 tokens max: 202 tokens	min: 164 tokens mean: 181.64 tokens max: 205 tokens	min: 164 tokens mean: 181.52 tokens max: 205 tokens

• Samples:

  sentence1	sentence2	negative
  B2M, MALAT1, SCAPER, TMSB4X, IFI35, GLTSCR2, JUNB, TMSB10, NAA20, PMAIP1, JUN, PTMA, HLA-C, ACTB, H3F3B, S100A4, EEF1A1, BTG1, IL32, DUSP1, HLA-A, HLA-B, VIM, FTH1, FAU, NACA, CD52, EEF1D, HNRNPA1, FOS, UBA52, FTL, HLA-E, ARHGDIB, EIF1, SELL, SRSF7, ARPC2, SP110, LTB, PPIA, LINC-PINT, VPS28, ANXA1, PFDN5, UBC, HMGB1, DAD1, SRSF5, CALM1 are the top expressed genes in this cell.	Cells expressing MALAT1, B2M, TMSB4X, JUNB, EEF1A1, TMSB10, PTMA, FTH1, TPT1, EIF1, HLA-A, NACA, FOS, JUN, HNRNPA1, FTL, ID2, DUSP1, HLA-C, PABPC1, UBC, SRSF5, KLF2, GNB2L1, HLA-B, TMEM66, FAU, GLTSCR2, TRAF3IP3, ZFP36L2, EEF1B2, BTF3, ACTB, EEF1D, CFL1, SERF2, IL32, CD52, CD53, CXCR4, NCL, NPM1, LTB, BRD2, PNRC1, RAC1, TSC22D3, ATP6V1G1, EIF4G2, NUCB2 often belong to the CD4 T cells lineage.	This transcriptomic profile — with genes like B2M, MALAT1, TMSB4X, ACTB, SNRPE, CEBPD, HLA-A, HLA-C, EEF1D, EEF1A1, CCL5, EIF1, CD52, RGS2, HLA-B, KLF6, NACA, TPT1, PFN1, PTMA, NPM1, TOMM7, IL2RG, ATP5G2, H3F3B, UBA52, S100A4, DUSP1, PABPC1, FAU, CFL1, UBC, FOS, PPIB, ACTG1, JUNB, S100A6, TRAF3IP3, GUK1, OST4, EIF4A2, HOPX, IL7R, GZMK, GNB2L1, LTB, GIMAP1, CCDC107, PRF1, FTH1 — resembles that of a CD8 T cells.
  Observed top genes: MALAT1, B2M, TMSB4X, TMSB10, YWHAB, EEF1A1, IL32, NKG7, GZMH, H3F3B, PPDPF, SH3BGRL3, S100A6, EEF1B2, GNB2L1, ACTB, FTH1, NACA, CCL5, EEF2, FXYD5, TMEM50A, CD52, LAPTM5, CD53, S100A10, DCAF8, UFC1, TRAF3IP3, IMMT, GCC2, ARPC2, PTMA, HINT1, SQSTM1, HLA-C, HLA-B, RBM3, SLC25A5, EEF1D, SRGN, IFITM1, TPT1, SRSF5, FOS, CALM1, HMOX2, CIRBP, CLPP, PRMT2.	This cell likely originates from the CD8 T cells family, based on expression of MALAT1, B2M, TMSB4X, RHOG, NKG7, HLA-C, CCL5, ACTB, HLA-A, HLA-B, TXNIP, TMSB10, EEF1A1, PTPRCAP, EIF1, PFN1, S100A4, FTH1, CFL1, CTSW, UCP2, NACA, SH3BGRL3, PTMA, CLIC1, HLA-DPB1, PRF1, FAU, ARHGDIB, TPT1, UBB, MYL12A, CST7, PPDPF, SSR2, IAH1, ARPC2, HLA-E, SLC25A6, CD3D, TPI1, UBC, CALM1, IDH2, H3F3B, C19orf43, FTL, CRELD2, CD52, S100A6.	CD4 T cells cells typically express genes such as: MALAT1, B2M, TMSB4X, TMSB10, EEF1A1, PTMA, FOS, JUN, ACTB, JUNB, CEBPB, TPT1, GCH1, EIF1, HLA-C, FAU, CD52, LPIN1, HLA-A, GNB2L1, FTH1, CFL1, BTG1, UBA52, CXCR4, NACA, PFN1, GLTSCR2, FTL, S100A4, ZFP36L2, BTF3, PABPC1, CD3D, LDHB, ACTG1, DNAJB1, YBX1, SERBP1, S100A6, CD247, EEF1B2, SATB1, TRAT1, CNBP, LTB, TMEM66, EEF1D, HSPA8, SLC2A3.
  This cell shows high expression of MALAT1, TMSB4X, B2M, EIF4A2, TPT1, TBCC, EEF1A1, PTMA, FOS, CHTF8, GNB2L1, HLA-C, FAU, NACA, JUNB, TMSB10, LTB, ACTB, EEF1D, DUSP1, HNRNPA1, GLTSCR2, JUN, TSC22D3, BTG1, UBC, HLA-B, CFL1, HLA-A, GAPDH, LDHB, EIF1, UBA52, EEF1B2, COX7C, HNRNPA2B1, PPIA, CD3D, NAP1L1, SRSF5, SERF2, H3F3B, ACTG1, ENO1, SH3BGRL3, MCL1, ZFP36L2, MGAT4A, NBEAL1, ARPC2, suggesting it is a CD4 T cells.	The genes MALAT1, TMSB4X, B2M, EEF1A1, JUNB, TMSB10, HLA-C, FTL, ACTB, UBA52, NPM1, GNB2L1, HLA-A, PTPRCAP, NACA, EIF1, PTMA, FAU, HNRNPA1, TPT1, FOS, ZFP36L2, HLA-E, EEF1D, IL32, IL7R, VIM, ATP5L, LDHB, MYL6, SRSF11, TXNIP, S100A6, BTG2, CXCR4, TUBA4A, DUSP1, HLA-B, C6orf48, SYPL1, TMEM66, FTH1, HSPA8, ARHGDIB, NAP1L1, BTG1, COMMD6, PSME1, CIB1, EIF4A1 are expressed in this cell, which is classified as a CD4 T cells.	Consistent with B cells function, genes like CD74, MALAT1, TMSB4X, B2M, HLA-DRA, SBDS, HLA-DPB1, PTMA, HLA-DRB1, LAPTM5, JUN, HLA-C, EEF1A1, FTH1, FOS, JUNB, HLA-DQA1, CFL1, UBE2I, OAZ1, CD79A, FTL, DUSP1, GNB2L1, HLA-B, ACTB, EEF1D, CIB1, EIF1, ARPC2, HLA-E, SLC25A6, MS4A1, ISCU, UBB, CD37, SH3BGRL3, HLA-A, HLA-DRB5, HLA-DPA1, HSPB1, FAU, NEAT1, PFN1, DDX5, H3F3B, ACTG1, UBA52, CD52, TMSB10 are expressed.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 32
• num_train_epochs: 2
• warmup_steps: 100

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 2
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 100
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	triplet_eval_scrna_cosine_accuracy
-1	-1	-	0.9290
0.0299	10	3.6988	-
0.0599	20	3.4047	-
0.0898	30	3.1924	-
0.1198	40	3.0103	-
0.1497	50	2.9363	-
0.1796	60	2.9156	-
0.2096	70	2.8106	-
0.2395	80	2.8403	-
0.2695	90	2.7849	-
0.2994	100	2.8646	-
0.3293	110	2.7653	-
0.3593	120	2.7846	-
0.3892	130	2.8744	-
0.4192	140	2.7813	-
0.4491	150	2.7164	-
0.4790	160	2.8228	-
0.5090	170	2.7669	-
0.5389	180	2.6674	-
0.5689	190	2.765	-
0.5988	200	2.7566	-
0.6287	210	2.6493	-
0.6587	220	2.7617	-
0.6886	230	2.6807	-
0.7186	240	2.7033	-
0.7485	250	2.6539	-
0.7784	260	2.6875	-
0.8084	270	2.6952	-
0.8383	280	2.6808	-
0.8683	290	2.6485	-
0.8982	300	2.6401	-
0.9281	310	2.6172	-
0.9581	320	2.6623	-
0.9880	330	2.6324	-
1.0180	340	2.6432	-
1.0479	350	2.6686	-
1.0778	360	2.7316	-
1.1078	370	2.8819	-
1.1377	380	2.8129	-
1.1677	390	2.4819	-
1.1976	400	2.9234	-
1.2275	410	2.938	-
1.2575	420	2.6774	-
1.2874	430	2.8483	-
1.3174	440	3.3175	-
1.3473	450	2.8162	-
1.3772	460	2.7896	-
1.4072	470	2.7999	-
1.4371	480	2.6654	-
1.4671	490	2.6718	-
1.4970	500	2.7074	-
1.5269	510	2.6552	-
1.5569	520	2.5986	-
1.5868	530	2.6024	-
1.6168	540	2.5522	-
1.6467	550	2.6396	-
1.6766	560	2.665	-
1.7066	570	2.6301	-
1.7365	580	2.5883	-
1.7665	590	2.6199	-
1.7964	600	2.4972	-
1.8263	610	2.6068	-
1.8563	620	2.5497	-
1.8862	630	2.6033	-
1.9162	640	2.6424	-
1.9461	650	2.6569	-
1.9760	660	2.5591	-

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.1.2
• Transformers: 4.57.1
• PyTorch: 2.9.1
• Accelerate: 1.11.0
• Datasets: 4.4.1
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}