Hugging Face's logo

JHyeok5
/
aewol-embedding-v1

Sentence Similarity
sentence-transformers
Safetensors
xlm-roberta
feature-extraction
dense
Generated from Trainer
dataset_size:6238
loss:TripletLoss
Eval Results (legacy)
text-embeddings-inference
Model card Files
xet
 Community

SentenceTransformer based on juyoungml/bge-m3-ko-v1.1

This is a sentence-transformers model finetuned from juyoungml/bge-m3-ko-v1.1. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: juyoungml/bge-m3-ko-v1.1
  • Maximum Sequence Length: 8192 tokens
  • Output Dimensionality: 1024 dimensions
  • Similarity Function: Cosine Similarity

Model Sources

Full Model Architecture 

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'XLMRobertaModel'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    '제주 항일운동가 손실 소식에 대해 알고 싶습니다.',
    '다. 항일ㆍ노동운동의 거목이 손실되자 조선인ㆍ일본인의 노동계가 합동으로 시신을 유리관(棺)에 안치, 오사카시 노동장(勞動葬)으로 성대히 엄수... (내용이 길어 일부 생략됨)',
    '머리가 되는 것이 아니다. 누구가 먼저 키를 잡는 수완과 자질을 발휘해 선점하느냐에 따라 도사공의 지위를 차지하게 된다는 말이다.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000, -0.9309, -0.9304],
#         [-0.9309,  1.0000,  0.9989],
#         [-0.9304,  0.9989,  1.0000]])

Evaluation

Metrics

Triplet

Metric Value
cosine_accuracy 0.4135

Training Details

Training Dataset

Unnamed Dataset

  • Size: 6,238 training samples
  • Columns: sentence_0, sentence_1, and sentence_2
  • Approximate statistics based on the first 1000 samples:
    sentence_0 sentence_1 sentence_2
    type string string string
    details
    • min: 8 tokens
    • mean: 20.85 tokens
    • max: 91 tokens
    • min: 8 tokens
    • mean: 216.35 tokens
    • max: 2850 tokens
    • min: 6 tokens
    • mean: 193.04 tokens
    • max: 2850 tokens
  • Samples:
    sentence_0 sentence_1 sentence_2
    할머니, 할아버지는 어떤 모습일까요? [삼 서끈 할망은 놀레 불르곡, 노 서끈 하르방은 운다.]
    삼 섞은 할머니는 노래 부르고, 노 섞은 할아버지는 운다. 여자는 인내력과 서둘지 않은 차분한 일 처리 능력이 남자에 비해서 앞선다는 말이다.    		 정부에서는 고인의 공훈을 기리어 1990년 광복절에 건국훈장 애족장을 추서하였다.
    제주도에서는 밥 먹는 일이 일과 어떤 연관이 있나요? 하고 다음 일을 할 수 있다. 어느 한 쪽이 늦어지면 그만큼 다음 일 착수가 늦어질 뿐만 아리나 일 처리도 미뤄지므로, 밥 먹는 일 자체가 곧 일을 차리는 것이 된다는 말이다.
    의미: 밥 먹음도 일 차림이다. 밥을 먹는 일은 평상시 정례화 된 사실이지만, 먹는 쪽이나 그 식사를 마련하는 쪽이 다 정해진 일과의 한 과정을 수행하고 있는 것이다. 그러니 먹는 사람은 먹어야 다음 일을 시작할 수 있고, 식사를 차려놓는 사람은 제 때에 먹어 버려야 뒤처리를 하고 다음 일을 할 수 있다. 어느 한 쪽이 늦어지면 그만큼 다음 일 착수가 늦어질 뿐만 아리나 일 처리도 미뤄지므로, 밥 먹는 일 자체가 곧 일을 차리는 것이 된다는 말이다.    		 나하고 둘이 둥실 사랑하자.`
    행원남당본풀이 이야기 들어보고 싶습니다. [행원남당본풀이]
    (myth)
    [신화] MYTH2.002 - 핵심어: 본풀이,행원남당본풀이    		 못하고 몸만 상하고 마는 현상을 지적하고 있다
  • Loss: TripletLoss with these parameters:
    {
    "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
    "triplet_margin": 5
}

Training Hyperparameters

Non-Default Hyperparameters

  • eval_strategy: steps
  • per_device_train_batch_size: 1
  • per_device_eval_batch_size: 1
  • fp16: True
  • multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand
  • overwrite_output_dir: False
  • do_predict: False
  • eval_strategy: steps
  • prediction_loss_only: True
  • per_device_train_batch_size: 1
  • per_device_eval_batch_size: 1
  • per_gpu_train_batch_size: None
  • per_gpu_eval_batch_size: None
  • gradient_accumulation_steps: 1
  • eval_accumulation_steps: None
  • torch_empty_cache_steps: None
  • learning_rate: 5e-05
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • max_grad_norm: 1
  • num_train_epochs: 3
  • max_steps: -1
  • lr_scheduler_type: linear
  • lr_scheduler_kwargs: {}
  • warmup_ratio: 0.0
  • warmup_steps: 0
  • log_level: passive
  • log_level_replica: warning
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • save_safetensors: True
  • save_on_each_node: False
  • save_only_model: False
  • restore_callback_states_from_checkpoint: False
  • no_cuda: False
  • use_cpu: False
  • use_mps_device: False
  • seed: 42
  • data_seed: None
  • jit_mode_eval: False
  • bf16: False
  • fp16: True
  • fp16_opt_level: O1
  • half_precision_backend: auto
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • local_rank: 0
  • ddp_backend: None
  • tpu_num_cores: None
  • tpu_metrics_debug: False
  • debug: []
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_prefetch_factor: None
  • past_index: -1
  • disable_tqdm: False
  • remove_unused_columns: True
  • label_names: None
  • load_best_model_at_end: False
  • ignore_data_skip: False
  • fsdp: []
  • fsdp_min_num_params: 0
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • fsdp_transformer_layer_cls_to_wrap: None
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • parallelism_config: None
  • deepspeed: None
  • label_smoothing_factor: 0.0
  • optim: adamw_torch_fused
  • optim_args: None
  • adafactor: False
  • group_by_length: False
  • length_column_name: length
  • project: huggingface
  • trackio_space_id: trackio
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • skip_memory_metrics: True
  • use_legacy_prediction_loop: False
  • push_to_hub: False
  • resume_from_checkpoint: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_private_repo: None
  • hub_always_push: False
  • hub_revision: None
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • include_inputs_for_metrics: False
  • include_for_metrics: []
  • eval_do_concat_batches: True
  • fp16_backend: auto
  • push_to_hub_model_id: None
  • push_to_hub_organization: None
  • mp_parameters:
  • auto_find_batch_size: False
  • full_determinism: False
  • torchdynamo: None
  • ray_scope: last
  • ddp_timeout: 1800
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • include_tokens_per_second: False
  • include_num_input_tokens_seen: no
  • neftune_noise_alpha: None
  • optim_target_modules: None
  • batch_eval_metrics: False
  • eval_on_start: False
  • use_liger_kernel: False
  • liger_kernel_config: None
  • eval_use_gather_object: False
  • average_tokens_across_devices: True
  • prompts: None
  • batch_sampler: batch_sampler
  • multi_dataset_batch_sampler: round_robin
  • router_mapping: {}
  • learning_rate_mapping: {}

Training Logs

Epoch Step Training Loss val_cosine_accuracy
0.0802 500 4.8733 0.4135

Framework Versions

  • Python: 3.12.12
  • Sentence Transformers: 5.2.0
  • Transformers: 4.57.3
  • PyTorch: 2.9.0+cu126
  • Accelerate: 1.12.0
  • Datasets: 4.0.0
  • Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss 

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}
Downloads last month
1
Safetensors
Model size
0.6B params
Tensor type
F32
·
Inference Providers NEW
Sentence Similarity
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for JHyeok5/aewol-embedding-v1

Base model

BAAI/bge-m3
Finetuned
juyoungml/bge-m3-ko-v1.1
Finetuned
(1)
this model

Papers for JHyeok5/aewol-embedding-v1

Evaluation results