jiwonyou0420/encoder-qa-finetuned-full

SentenceTransformer based on jiwonyou0420/MNLP_M2_document_encoder

This is a sentence-transformers model finetuned from jiwonyou0420/MNLP_M2_document_encoder. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: jiwonyou0420/MNLP_M2_document_encoder
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("jiwonyou0420/encoder-qa-finetuned-full")
# Run inference
sentences = [
    "A physics student wants to know how to calculate the trajectory of a relativistic particle moving at a constant velocity. The particle's initial position is (0, 0) and its velocity is v = 0.8c, where c is the speed of light. The student also knows that the particle experiences no forces during its motion. What is the equation for the particle's trajectory as a function of time in terms of its initial position and velocity? How long does it take for the particle to reach a point where x = 2 meters?",
    'where x ( t ) is the position of the particle at time t, x0 is the initial position ( 0 in this case ), and v is the velocity ( 0. 8c ). now, we want to find the time it takes for the particle to reach a point where x = 2 meters. to do this, we can set x ( t ) = 2 and solve for t : 2 = 0 + ( 0. 8c ) * t t = 2 / ( 0. 8c ) since c is the speed of light, which is approximately 3 x 10 ^ 8 meters per second, we can substitute this value',
    'the ferroelectric behavior of a crystal is highly dependent on external factors such as pressure and temperature. ferroelectric materials exhibit spontaneous electric polarization that can be reversed by an external electric field. this behavior arises due to the displacement of ions within the crystal lattice, leading to the formation of electric dipoles. the ferroelectric properties of a crystal can be altered by changes in external pressure and temperature, which affect the crystal lattice and the stability of the polarized state. 1. effect of temperature : the ferroelectric behavior of a crystal is strongly influenced by temperature. as the temperature increases, the thermal energy causes',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

• Size: 173,416 training samples
• Columns: sentence_0, sentence_1, and label
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1	label
  type	string	string	float
  details	min: 15 tokens mean: 47.28 tokens max: 318 tokens	min: 3 tokens mean: 115.26 tokens max: 142 tokens	min: 0.0 mean: 0.49 max: 1.0

• Samples:

  sentence_0	sentence_1	label
  What is the effect of dark matter on the temperature anisotropies in the cosmic microwave background radiation? Provide a quantitative analysis to determine the magnitude of this effect and explain how it can be detected through observations of the CMB.	planck satellite have provided high - resolution measurements of the cmb temperature anisotropies, allowing for precise constraints on the dark matter density and other cosmological parameters. in summary, dark matter affects the temperature anisotropies in the cmb by influencing the formation of large - scale structures and altering the power spectrum of the temperature fluctuations. the magnitude of this effect can be determined through a quantitative analysis of the power spectrum, and the presence of dark matter can be detected through observations of the cmb temperature anisotropies.	1.0
  How can the presence of monopoles affect the behavior of strings in string theory, and how can this be used to explain some unexplained phenomena in particle physics?	spin states of individual electrons in quantum dots, paving the way for the development of scalable quantum computing architectures. 2. quantum dot arrays : researchers have successfully created arrays of quantum dots that can be used to perform quantum operations. these arrays can be used as a platform for implementing quantum error correction codes, which are essential for building fault - tolerant quantum computers. 3. coherent coupling : coherent coupling between quantum dots has been demonstrated, allowing for the transfer of quantum information between qubits. this is a crucial step towards building large - scale quantum computers, as it enables the creation of quantum gates and entangled states. 4. integration with	0.0
  A physics student is investigating the flow properties of a non-Newtonian fluid. The student has data on the fluid's shear stress vs. shear rate behavior, and wants to understand its rheological properties. Can the student determine if the fluid is shear-thinning, shear-thickening, or a Bingham plastic? If so, what is the fluid's behavior index or yield stress?	yes, the student can determine if the fluid is shear - thinning, shear - thickening, or a bingham plastic by analyzing the data on the fluid ' s shear stress ( τ ) vs. shear rate ( γ ) behavior. 1. shear - thinning fluid : if the fluid ' s viscosity decreases with increasing shear rate, it is a shear - thinning fluid ( also known as pseudoplastic fluid ). in this case, the fluid ' s behavior index ( n ) will be less than 1. the relationship between shear stress and shear rate can be described by the power - law model : τ = k *	1.0

• Loss: CosineSimilarityLoss with these parameters:

  {
      "loss_fct": "torch.nn.modules.loss.MSELoss"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• num_train_epochs: 1
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• tp_size: 0
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	Training Loss
0.0923	500	0.0651
0.1845	1000	0.0457
0.2768	1500	0.0388
0.3690	2000	0.0374
0.4613	2500	0.0375
0.5535	3000	0.0342
0.6458	3500	0.0345
0.7380	4000	0.033
0.8303	4500	0.0313
0.9225	5000	0.0325

Framework Versions

• Python: 3.12.8
• Sentence Transformers: 3.4.1
• Transformers: 4.51.3
• PyTorch: 2.5.1+cu124
• Accelerate: 1.3.0
• Datasets: 3.2.0
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}