GSR-608001/avvaiyar-embedding-v2

SentenceTransformer based on intfloat/multilingual-e5-small

This is a sentence-transformers model finetuned from intfloat/multilingual-e5-small. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: intfloat/multilingual-e5-small
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 384 tokens
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("GSR-608001/avvaiyar-embedding-v2")
# Run inference
sentences = [
    "I have been failing to hate any desire for lust and it's affecting me",
    '\nAathichoodi Ethical Wisdom\n\nVerse: மோகத்தை முனி\n\nExplanation:\nCultivate a disciplined aversion to transient desires and obsessive attachments that cloud the intellect. The pursuit of fleeting pleasures or material accumulation often leads to a state of psychological bondage, where one’s happiness is dependent on external, unstable factors. By transcending these base impulses, an individual achieves a level of emotional independence and cognitive freedom. This detachment is not a rejection of the world but a strategic prioritization of enduring growth over the ephemeral distractions of sensory indulgence and irrational cravings.\n',
    '\nAathichoodi Ethical Wisdom\n\nVerse: இடம்பட வீடு எடேல்\n\nExplanation:\nThis ethical guideline promotes the philosophy of essentialism and the rejection of ostentatious consumption. In an age of environmental crisis and hyper-consumerism, it serves as a mandate for sustainable living and the prudent management of personal resources and physical space. By limiting one’s footprint and resisting the urge for excessive display, an individual aligns their lifestyle with ecological responsibility and internal contentment. True prosperity is framed here not as the accumulation of vast territory or luxury, but as the mastery of living meaningfully and efficiently within optimal, functional boundaries that do not infringe upon the needs of the collective.\n',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

• Size: 725 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 9 tokens mean: 20.24 tokens max: 45 tokens	min: 108 tokens mean: 154.41 tokens max: 181 tokens

• Samples:

  sentence_0	sentence_1
  What actions or attitudes should be avoided if one aims to truly practice structural support, communal stability, and selfless service?	Aathichoodi Ethical Wisdom Verse: ஙப் போல் வளை Explanation: The philosophical depth of this instruction lies in the metaphor of structural support and selfless inclusivity. Just as a specific foundational element may exist primarily to anchor and preserve its entire lineage without seeking individual prominence, an ethical actor must practice proactive empathy to protect and uplift their community. In modern organizational systems, this translates to servant leadership and the "invisible" work of social glue—being flexible enough to accommodate the needs of others while remaining firm in the commitment to collective welfare. It teaches that true strength is found in the ability to yield for the sake of communal stability and the preservation of shared heritage.
  I have been failing to don't sin and it's affecting me	Aathichoodi Ethical Wisdom Verse: தீவினை அகற்று Explanation: Ethical risk management demands the active identification and systematic removal of malevolent actions and thoughts from one’s life. Understanding the law of causality implies that destructive deeds inevitably lead to negative externalities, both for the perpetrator and the collective environment. By consciously desisting from harmful behaviors and refining one’s moral compass, an individual protects their psychological well-being and ensures they do not become a source of disorder or suffering within the social fabric.
  I have been failing to don't defame the divine and it's affecting me	Aathichoodi Ethical Wisdom Verse: தெய்வம் இகழேல் Explanation: Cultivate a profound respect for the transcendent and the metaphysical frameworks that provide moral structure to human existence. To defame the divine is to undermine the collective sanctity and the psychological anchors that guide a society toward humility and higher purpose. In a modern context, this encourages recognizing the limits of human ego and preserving the integrity of spiritual or existential ideals that foster social cohesion and individual mental well-being, ensuring that cynicism does not erode the foundational values of a community.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• num_train_epochs: 4
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 4
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Framework Versions

• Python: 3.11.9
• Sentence Transformers: 3.0.1
• Transformers: 4.41.2
• PyTorch: 2.5.1+cu121
• Accelerate: 1.12.0
• Datasets: 4.5.0
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}