along26/mpnet-base-manglish-triplet

MPNet base trained on Manglish triplets

This is a sentence-transformers model finetuned from microsoft/mpnet-base. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: microsoft/mpnet-base
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'MPNetModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("along26/mpnet-base-manglish-triplet")
# Run inference
sentences = [
    'A string of length 10 m and mass 0.01 kg is fixed at both ends. The wave function of the string is given as y(x,t) = 2sin(3x - 4t) where x and y are in meters and t is in seconds. \n\nWhat is the tension in the string?',
    'Seutas tali panjang 10 m dan berjisim 0.01 kg diikat pada kedua-dua hujungnya. Fungsi gelombang rentetan diberikan sebagai y(x,t) = 2sin(3x - 4t) dengan x dan y dalam meter dan t dalam saat.\n\nApakah ketegangan dalam tali?',
    "How has Najib Razak's handling of the economy been criticized, and what impact has this had on his political standing?",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000, -0.7525,  0.9936],
#         [-0.7525,  1.0000, -0.7687],
#         [ 0.9936, -0.7687,  1.0000]])

Evaluation

Metrics

Triplet

• Datasets: manglish-dev and manglish-test
• Evaluated with TripletEvaluator

Metric	manglish-dev	manglish-test
cosine_accuracy	0.0	0.0

Training Details

Training Dataset

Unnamed Dataset

• Size: 10,000 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 12 tokens mean: 219.95 tokens max: 512 tokens	min: 17 tokens mean: 263.51 tokens max: 512 tokens	min: 15 tokens mean: 236.31 tokens max: 512 tokens

• Samples:

  anchor	positive	negative
  Consider a graph with 8 vertices and 12 edges. Determine if the graph contains a perfect matching. If it does, provide one example of a perfect matching. If it does not, explain why a perfect matching is not possible.	Pertimbangkan graf dengan 8 bucu dan 12 tepi. Tentukan sama ada graf mengandungi padanan sempurna. Jika ya, berikan satu contoh padanan yang sempurna. Jika tidak, jelaskan mengapa padanan yang sempurna tidak dapat dilakukan.	The 1MDB scandal and the corruption charges against former Malaysian Prime Minister Najib Razak offer several important lessons for Malaysia and other countries: 1. Stronger checks and balances: The 1MDB scandal highlighted the need for stronger checks and balances in government agencies and institutions. Malaysia should consider implementing additional measures to prevent the misuse of power and public funds, including more robust auditing and oversight mechanisms, as well as stronger whistleblower protections. 2. Transparency and accountability: The lack of transparency and accountability surrounding 1MDB contributed to the scandal. Malaysia should prioritize transparency in government operations, including procurement processes and financial transactions. Implementing measures such as open data initiatives and requiring greater disclosure from government-linked companies could help promote accountability and reduce opportunities for corruption. 3. Strengthening law enforcement: The...
  What is the probability of flipping a fair coin three times and getting exactly two heads in a row?	Apakah kebarangkalian membalikkan syiling saksama tiga kali dan mendapat tepat dua kepala berturut-turut?	Why is corruption so rampant in Malaysia, with politicians and government officials often caught engaging in unethical practices?
  Why have there been allegations of corruption and mismanagement in Malaysia's state-owned enterprises, and what measures have been taken to address these issues?	Mengapa terdapat tuduhan rasuah dan salah urus dalam perusahaan milik kerajaan Malaysia, dan apakah langkah-langkah yang telah diambil untuk menangani isu-isu ini?	What is the pKa value of acetic acid, and how does it affect its acid strength when compared to other organic acids such as citric acid or benzoic acid? Provide an explanation for your answer using acid-base reaction principles in organic chemistry.

• Loss: TripletLoss with these parameters:

  {
      "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
      "triplet_margin": 5
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 1,000 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 12 tokens mean: 222.11 tokens max: 512 tokens	min: 15 tokens mean: 267.79 tokens max: 512 tokens	min: 12 tokens mean: 237.3 tokens max: 512 tokens

• Samples:

  anchor	positive	negative
  In a set of 50 data points, the mean is 70 and the standard deviation is 10. If an outlier is defined as any data point that is more than three standard deviations away from the mean, how many outliers are there in the data set?	Dalam set 50 titik data, min ialah 70 dan sisihan piawai ialah 10. Jika outlier ditakrifkan sebagai mana-mana titik data yang jauh lebih daripada tiga sisihan piawai daripada min, berapakah jumlah outlier yang terdapat dalam set data?	There is still a shortage of affordable housing for the lower and middle-income groups in Malaysia due to several factors, including rapid urbanization, population growth, and rising housing prices. According to the World Bank, Malaysia's urbanization rate reached 76% in 2019, and the country's population is expected to grow to 36 million by 2040, putting pressure on the housing market. The Malaysian government has implemented several initiatives to address the issue of affordable housing, including the launch of the National Affordable Housing Policy (MyAhp) in 2019, which aims to provide affordable housing to 1.3 million households by 2030. However, there is still a significant shortage of affordable housing. One of the main reasons for the shortage is the rising housing prices, which have outpaced income growth. According to the Khazanah Research Institute, house prices in Malaysia have increased by 9.8% per year from 2000 to 2016, while household income has only grown by 3.9% per...
  Why have some analysts argued that the corruption charges against Najib Razak represent a broader problem of endemic corruption in Malaysian politics, and what steps can be taken to address this issue?	Mengapa beberapa penganalisis berhujah bahawa tuduhan rasuah terhadap Najib Razak mewakili masalah rasuah endemik yang lebih luas dalam politik Malaysia, dan apakah langkah yang boleh diambil untuk menangani isu ini?	What is the current understanding of the large-scale structure of the universe, and what evidence supports this model?
  How has Malaysia's political landscape changed since the 2018 general election, which saw the ouster of Najib Razak and his UMNO-led coalition, and what challenges does the new government face in combating corruption?	Bagaimanakah landskap politik Malaysia berubah sejak pilihan raya umum 2018, yang menyaksikan penyingkiran Najib Razak dan gabungan pimpinan UMNOnya, dan apakah cabaran yang dihadapi kerajaan baharu dalam memerangi rasuah?	Snell's Law states that the ratio of the sine of the angle of incidence to the sine of the angle of refraction is equal to the ratio of the refractive indices of the two media. Mathematically, it can be written as: n1 * sin(θ1) = n2 * sin(θ2) where n1 and n2 are the refractive indices of the first and second media, and θ1 and θ2 are the angles of incidence and refraction, respectively. In this case, the ray of light travels from water (n1 = 1.33) into air (n2 = 1.00), and the incident angle is 60 degrees (θ1 = 60°). We want to find the angle of refraction (θ2). 1.33 * sin(60°) = 1.00 * sin(θ2) sin(60°) = 0.866, so: 1.33 * 0.866 = sin(θ2) 1.152 = sin(θ2) Now, we need to find the inverse sine (arcsin) of 1.152 to get θ2. However, the sine function has a maximum value of 1, so there is no real solution for this problem. This means that the ray of light does not refract into the air but instead undergoes total internal reflection within the water.

• Loss: TripletLoss with these parameters:

  {
      "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
      "triplet_margin": 5
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 4
• gradient_accumulation_steps: 4
• warmup_ratio: 0.1
• fp16: True
• dataloader_pin_memory: False

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 4
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 4
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: False
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss	manglish-dev_cosine_accuracy	manglish-test_cosine_accuracy
-1	-1	-	-	0.1400	-
0.08	50	1.6398	-	-	-
0.16	100	0.0828	-	-	-
0.24	150	0.0358	-	-	-
0.32	200	0.0113	0.0	0.0	-
0.4	250	0.0167	-	-	-
0.48	300	0.0	-	-	-
0.56	350	0.0	-	-	-
0.64	400	0.0	0.0	0.0	-
0.72	450	0.0	-	-	-
0.8	500	0.0	-	-	-
0.88	550	0.0007	-	-	-
0.96	600	0.0699	0.0074	0.0	-
1.04	650	0.0231	-	-	-
1.12	700	0.0158	-	-	-
1.2	750	0.0107	-	-	-
1.28	800	0.0039	0.0	0.0	-
1.3600	850	0.0	-	-	-
1.44	900	0.0	-	-	-
1.52	950	0.0114	-	-	-
1.6	1000	0.0115	0.0	0.0	-
1.6800	1050	0.0	-	-	-
1.76	1100	0.0025	-	-	-
1.8400	1150	0.0042	-	-	-
1.92	1200	0.0012	0.0	0.0	-
2.0	1250	0.0	-	-	-
2.08	1300	0.0	-	-	-
2.16	1350	0.0	-	-	-
2.24	1400	0.0064	0.0	0.0	-
2.32	1450	0.0	-	-	-
2.4	1500	0.0	-	-	-
2.48	1550	0.0	-	-	-
2.56	1600	0.0	0.0	0.0	-
2.64	1650	0.0	-	-	-
2.7200	1700	0.0089	-	-	-
2.8	1750	0.0	-	-	-
2.88	1800	0.0	0.0	0.0	-
2.96	1850	0.0	-	-	-
-1	-1	-	-	-	0.0000

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.1.2
• Transformers: 4.57.1
• PyTorch: 2.8.0+cu126
• Accelerate: 1.11.0
• Datasets: 4.0.0
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}