Hugging Face's logo

trongvox
/
Phobert-Sentence

Sentence Similarity
sentence-transformers
Safetensors
roberta
feature-extraction
Generated from Trainer
dataset_size:11347
loss:MultipleNegativesRankingLoss
Model card Files
xet
 Community

SentenceTransformer based on vinai/phobert-base

This is a sentence-transformers model finetuned from vinai/phobert-base. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: vinai/phobert-base
  • Maximum Sequence Length: 128 tokens
  • Output Dimensionality: 768 dimensions
  • Similarity Function: Cosine Similarity

Model Sources

Full Model Architecture 

SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: RobertaModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("trongvox/Phobert-Sentence")
# Run inference
sentences = [
    'Noi tieng ve do lau doi va huong vi mon an nay o Ha Noi thi phai ke den hang Banh Duc Nong Thanh Tung. Banh o day hap dan o do deo dai cua bot, thit nam du day va nem nem vua mieng. Khi phuc vu, mon an nong sot toa ra mui huong thom lung tu bot, hanh phi, nuoc mam. Mon banh duc o day duoc chan ngap nuoc mam pha loang vi ngot, hoi man man, co thit bam voi nam meo va rat nhieu hanh kho da phi vang.Mon banh duc o Banh Duc Nong Thanh Tung duoc chan ngap nuoc mam pha loang vi ngot, hoi man man, co thit bam voi nam meo va rat nhieu hanh kho da phi vang. Cach an nay hoi giong voi mon banh gio chan nuoc mam thit bam o quan pho chua Lang Son gan cho Ban Co. La mon qua an nhe nhang, vua du lung lung bung, co ve dan da nen rat nhieu nguoi them them, nho nho. Banh duc nong Ha Noi o day khong bi pha them bot dau xanh nen van giu nguyen duoc huong vi dac trung. Dac biet, phan nhan con duoc tron them mot it cu dau xao tren ngon lua lon nen giu duoc do ngot gion.THONG TIN LIEN HE:Dia chi: 112 Truong Dinh, Quan Hai Ba Trung, Ha NoiGio mo cua: 10:00 - 21:00Dia diem chat luong: 4.7/5 (14 danh gia tren Google)\n Chi duong Danh gia Google',
    'Banh Duc',
    'Banh bi do',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

  • Size: 11,347 training samples
  • Columns: sentence_0 and sentence_1
  • Approximate statistics based on the first 1000 samples:
    sentence_0 sentence_1
    type string string
    details
    • min: 73 tokens
    • mean: 127.74 tokens
    • max: 128 tokens
    • min: 3 tokens
    • mean: 8.16 tokens
    • max: 24 tokens
  • Samples:
    sentence_0 sentence_1
    Mamadeli la mot dia chi giup ban giai quyet con them com ga, mi y chuan vi nhat. Nhan vien tai quan nay kha de chiu va chieu khach. Mot suat com ga ta bao gom mot phan com mem, thit ga ta xe thom phuc va dia nuoc mam gung chan voi sot trung rat dam da.Giua long Sai Gon hoa le lai co huong vi cua mon com ga nuc tieng thi con dieu gi khien ban ban khoan ma khong thuong thuc nhi. Thuc don phong phu, gia ca phai chang voi huong vi mon an hoan hao dung vi hap dan la li do giup quan thu hut duoc dong dao khach hang ghe toi thuong xuyen.

    Ngoai ra, voi cach trinh bay mon an day bat mat va mau sac chac chan cac thuc khach khi den day se khong the roi mat khoi mon an dau. Team thich song ao tung chao nghe toi day chac hao huc lam vi do an vua ngon, vua co hinh de song ao chat luong.Va khien ai cung thom them ghen ti khi ban co co hoi duoc thu va trai nghiem o Mamadeli do. Neu ban muon tan huong tai nha thi hay yen tam, Mamadeli hien tai da co mat tren cac app giao hang, cac ban co the theo doi...    		 Mamadeli - Com ga & Mi y
    Nguyen lieu:Thit heo xay 300 gr Toi bam 2 muong ca phe Hanh tim bam 2 muong ca phe Gung bam 1 muong ca phe Nuoc mam 1/2 muong canh Nuoc tuong 1 muong canh Bot nang 2 muong canh Giam an 2 muong canh Tuong ca 3 muong canh Dau an 2 muong canh Duong 4 muong canh Muoi 1/4 muong canhCach che bien Thit vien kho chua ngotUop thitBan uop thit voi 2 muong ca phe toi bam, 2 muong ca phe hanh tim, 1 muong ca phe gung bam, 1/4 muong ca phe muoi, 1/2 muong canh nuoc mam, 1 muong canh nuoc tuong, 2 muong canh bot nang.Sau do, ban tron deu de cac gia vi ngam vao nhau va uop khoang 15 phut.
    Vo vien va chien thitBan vo thit thanh tung vien vua an.Ban dun nong 2 muong canh dau an o lua vua. Khi dau soi, ban cho thit vao va chien vang deu 2 mat.
    Kho thitBan cho vao chao 4 muong canh duong, 2 muong canh giam an, 3 muong canh tuong ca va 4 muong canh nuoc loc roi dao deu.Ban rim phan nuoc sot voi thit vien 15 phut sau do tat bep va cho ra dia.
    Thanh phamThit vien mem, thom, vua an cung voi nuoc sot chua chu...    		 Thit vien kho chua ngot
    Nguyen lieu:1kg oc1 cu gungHanh khoToi, otSa teNuoc mam, bot ngot, duong...Cach lam:Oc giac khi mua ve, ban cung dem rua sach, roi ngam voi nuoc vo gao co cat them voilat ot trong 3 tieng de oc nhanh nha chat ban ra.Gung ban dem cao vo rua sach, bam nho.Hanh kho, toi boc sach vo. Hanh kho ban thai lat mong, con toi thi bam nhuyen.Ot tuoi rua sach, thai lat.Sau khi ngam xong, ban dem oc giac luoc voi nuoc co cho them vai lat gung hoac sa dap dap. Khi oc chin, ban lay thit oc ra cat lat va de ra dia. Dat chao len bep, cho dau an vao, khi dau soi ban cho hanh kho va toi vao phi thom. Tiep den, ban cho vao 3 muong sa te, ot cat lat, dao deu tay. Dao khoang 5 phut, ban cho oc vao deu roi nem nem voi nuoc mam, duong, bot ngot sao cho vua khau vi. Xao oc khoang 10 phut nua thi tat bep.Vay la hoan thanh mon an roi, gio day ban chi can cho mon an ra dia va cho them vai soi rau ram len tren la xong! Oc giac xao sa te
  • Loss: MultipleNegativesRankingLoss with these parameters:
    {
    "scale": 20.0,
    "similarity_fct": "cos_sim"
}

Training Hyperparameters

Non-Default Hyperparameters

  • per_device_train_batch_size: 16
  • per_device_eval_batch_size: 16
  • multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand
  • overwrite_output_dir: False
  • do_predict: False
  • eval_strategy: no
  • prediction_loss_only: True
  • per_device_train_batch_size: 16
  • per_device_eval_batch_size: 16
  • per_gpu_train_batch_size: None
  • per_gpu_eval_batch_size: None
  • gradient_accumulation_steps: 1
  • eval_accumulation_steps: None
  • torch_empty_cache_steps: None
  • learning_rate: 5e-05
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • max_grad_norm: 1
  • num_train_epochs: 3
  • max_steps: -1
  • lr_scheduler_type: linear
  • lr_scheduler_kwargs: {}
  • warmup_ratio: 0.0
  • warmup_steps: 0
  • log_level: passive
  • log_level_replica: warning
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • save_safetensors: True
  • save_on_each_node: False
  • save_only_model: False
  • restore_callback_states_from_checkpoint: False
  • no_cuda: False
  • use_cpu: False
  • use_mps_device: False
  • seed: 42
  • data_seed: None
  • jit_mode_eval: False
  • use_ipex: False
  • bf16: False
  • fp16: False
  • fp16_opt_level: O1
  • half_precision_backend: auto
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • local_rank: 0
  • ddp_backend: None
  • tpu_num_cores: None
  • tpu_metrics_debug: False
  • debug: []
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_prefetch_factor: None
  • past_index: -1
  • disable_tqdm: False
  • remove_unused_columns: True
  • label_names: None
  • load_best_model_at_end: False
  • ignore_data_skip: False
  • fsdp: []
  • fsdp_min_num_params: 0
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • fsdp_transformer_layer_cls_to_wrap: None
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • deepspeed: None
  • label_smoothing_factor: 0.0
  • optim: adamw_torch
  • optim_args: None
  • adafactor: False
  • group_by_length: False
  • length_column_name: length
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • skip_memory_metrics: True
  • use_legacy_prediction_loop: False
  • push_to_hub: False
  • resume_from_checkpoint: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_private_repo: None
  • hub_always_push: False
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • include_inputs_for_metrics: False
  • include_for_metrics: []
  • eval_do_concat_batches: True
  • fp16_backend: auto
  • push_to_hub_model_id: None
  • push_to_hub_organization: None
  • mp_parameters:
  • auto_find_batch_size: False
  • full_determinism: False
  • torchdynamo: None
  • ray_scope: last
  • ddp_timeout: 1800
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • dispatch_batches: None
  • split_batches: None
  • include_tokens_per_second: False
  • include_num_input_tokens_seen: False
  • neftune_noise_alpha: None
  • optim_target_modules: None
  • batch_eval_metrics: False
  • eval_on_start: False
  • use_liger_kernel: False
  • eval_use_gather_object: False
  • average_tokens_across_devices: False
  • prompts: None
  • batch_sampler: batch_sampler
  • multi_dataset_batch_sampler: round_robin

Training Logs

Epoch Step Training Loss
0.7042 500 0.9125
1.4085 1000 0.2277
2.1127 1500 0.1527
2.8169 2000 0.1009

Framework Versions

  • Python: 3.10.12
  • Sentence Transformers: 3.3.1
  • Transformers: 4.47.1
  • PyTorch: 2.5.1+cu121
  • Accelerate: 1.2.1
  • Datasets: 3.2.0
  • Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss 

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
Downloads last month
2
Safetensors
Model size
0.1B params
Tensor type
F32
·
Inference Providers NEW
Sentence Similarity
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for trongvox/Phobert-Sentence

Base model

vinai/phobert-base
Finetuned
(190)
this model

Papers for trongvox/Phobert-Sentence