README.md

metadata

tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - dense
  - generated_from_trainer
  - dataset_size:20000
  - loss:MultipleNegativesRankingLoss
base_model: sentence-transformers/all-MiniLM-L12-v2
widget:
  - source_sentence: Rules regarding Hearing procedures and due process (policy)
    sentences:
      - >-
        Graduation requires fulfillment of all academic requirements, clearance
        of financial obligations, completion of residency, and submission of
        necessary documents as listed in the graduation checklist.
      - >-
        Students charged with offenses are entitled to notice, access to
        evidence, the opportunity to answer, and representation during hearings.
        Hearing committees are constituted according to the Manual to ensure
        impartiality.
      - >-
        Student organizations must secure registration and recognition from the
        Office of Student Affairs. Official representation in external
        activities requires written authorization and compliance with OSA
        procedures.
  - source_sentence: Procedure for Use of campus facilities and hours
    sentences:
      - >-
        An NC is permanent and assigned when a student fails to take required
        examinations or submit essential academic requirements; it carries no
        credit and cannot be converted to a passing grade.
      - >-
        Scholarship eligibility depends on academic performance and other
        criteria; renewal requires compliance with the conditions stated in the
        scholarship guidelines and submission of required documents.
      - >-
        Access to campus facilities is subject to posted hours of operation and
        may require prior booking or authorization. Misuse or vandalism of
        facilities results in disciplinary measures.
  - source_sentence: Official rule on Incomplete (4.0) and removal
    sentences:
      - >-
        Use of laboratories requires compliance with safety protocols, proper
        attire, and authorization from the laboratory-in-charge. Unauthorized
        access and misuse of equipment will result in disciplinary action.
      - >-
        Use of laboratories requires compliance with safety protocols, proper
        attire, and authorization from the laboratory-in-charge. Unauthorized
        access and misuse of equipment will result in disciplinary action.
      - >-
        An INC (4.0) must be removed within one year by completing outstanding
        requirements; failure to remove the INC within the allowed period
        converts it to a failing grade (5.0).
  - source_sentence: Rules regarding Special examination policy
    sentences:
      - >-
        Graduation requires fulfillment of all academic requirements, clearance
        of financial obligations, completion of residency, and submission of
        necessary documents as listed in the graduation checklist.
      - >-
        Special examinations may be requested within the timeframe specified in
        the manual and require payment of the applicable special exam fee,
        clearance of financial obligations, and approval from the Program Chair.
      - >-
        Scholarship eligibility depends on academic performance and other
        criteria; renewal requires compliance with the conditions stated in the
        scholarship guidelines and submission of required documents.
  - source_sentence: Prohibited clothing on non-uniform days policy (policy)
    sentences:
      - >-
        Students must complete Physical Education and NSTP/CWTS within the
        prescribed curriculum period. Failure to complete these may delay
        graduation or limit enrollment options as detailed in the Manual.
      - >-
        Final grades are computed using the progressive formulas in the manual:
        Period Grade (PG), Midterm Grade (MG), and Final Grade (FG) which
        combine class standing and examination components according to
        prescribed weights.
      - >-
        Indecent prints, excessively revealing or tight clothing, mini-skirts,
        bare midriffs, transparent or torn pants, shorts, caps or hats inside
        classrooms, and slippers/clogs are not permitted during non-uniform
        days.
pipeline_tag: sentence-similarity
library_name: sentence-transformers

SentenceTransformer based on sentence-transformers/all-MiniLM-L12-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L12-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L12-v2
• Maximum Sequence Length: 128 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Prohibited clothing on non-uniform days policy (policy)',
    'Indecent prints, excessively revealing or tight clothing, mini-skirts, bare midriffs, transparent or torn pants, shorts, caps or hats inside classrooms, and slippers/clogs are not permitted during non-uniform days.',
    'Final grades are computed using the progressive formulas in the manual: Period Grade (PG), Midterm Grade (MG), and Final Grade (FG) which combine class standing and examination components according to prescribed weights.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.9320, 0.0963],
#         [0.9320, 1.0000, 0.0644],
#         [0.0963, 0.0644, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 20,000 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 5 tokens mean: 10.57 tokens max: 23 tokens	min: 29 tokens mean: 38.92 tokens max: 53 tokens

• Samples:

  sentence_0	sentence_1
  Official rule on Appeals and grievance procedures	Students may file appeals or grievances following the process specified in the Manual. Timelines, required forms, and steps for escalation are provided in the relevant section.
  Explain the No Credit (NC) policy.	An NC is permanent and assigned when a student fails to take required examinations or submit essential academic requirements; it carries no credit and cannot be converted to a passing grade.
  Official rule on Grooming and appearance standards	Students must maintain neat and conservative grooming. Unnatural loud hair coloring, multiple visible earrings beyond one per ear, facial tattoos, and excessive piercings are disallowed. Violations may be referred to the Office of Student Affairs.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.8	500	0.6267
1.6	1000	0.614
2.4	1500	0.6082

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.1.2
• Transformers: 4.57.1
• PyTorch: 2.8.0+cu126
• Accelerate: 1.11.0
• Datasets: 4.0.0
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}