Hugging Face's logo

dkqjrm
/
lora_model

Sentence Similarity
sentence-transformers
Safetensors
feature-extraction
dense
Generated from Trainer
dataset_size:1310129
loss:MultipleNegativesRankingLoss
Model card Files
xet
Community

SentenceTransformer based on unsloth/Qwen3-Embedding-0.6B

This is a sentence-transformers model finetuned from unsloth/Qwen3-Embedding-0.6B on the olive-product dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: unsloth/Qwen3-Embedding-0.6B
  • Maximum Sequence Length: 512 tokens
  • Output Dimensionality: 1024 dimensions
  • Similarity Function: Cosine Similarity
  • Training Dataset:

Model Sources

Full Model Architecture 

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'PeftModelForFeatureExtraction'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("dkqjrm/lora_model")
# Run inference
sentences = [
    '랑방 랑방 메리미 EDP 50ml',
    '浪凡 EDP',
    'マリーミー EDP',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.6852, 0.6339],
#         [0.6852, 1.0000, 0.3744],
#         [0.6339, 0.3744, 1.0000]])

Training Details

Training Dataset

olive-product

  • Dataset: olive-product at 8d1f081
  • Size: 1,310,129 training samples
  • Columns: anchor and positive
  • Approximate statistics based on the first 1000 samples:
    anchor positive
    type string string
    details
    • min: 16 tokens
    • mean: 25.67 tokens
    • max: 47 tokens
    • min: 2 tokens
    • mean: 7.19 tokens
    • max: 38 tokens
  • Samples:
    anchor positive
    베르사체 베르사체 브라이트 크리스탈 50ml 택1 베르사체 브라이트 크리스탈 50ml 1
    베르사체 베르사체 브라이트 크리스탈 50ml 택1 베르사체 브라이트
    베르사체 베르사체 브라이트 크리스탈 50ml 택1 베르사체 크리스탈
  • Loss: MultipleNegativesRankingLoss with these parameters:
    {
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false
}

Training Hyperparameters

Non-Default Hyperparameters

  • per_device_train_batch_size: 32
  • gradient_accumulation_steps: 4
  • learning_rate: 3e-05
  • num_train_epochs: 2
  • lr_scheduler_type: constant_with_warmup
  • warmup_ratio: 0.03
  • fp16: True
  • push_to_hub: True
  • hub_model_id: dkqjrm/qwen3-embedding-olive-lora
  • batch_sampler: no_duplicates

All Hyperparameters

Click to expand
  • overwrite_output_dir: False
  • do_predict: False
  • eval_strategy: no
  • prediction_loss_only: True
  • per_device_train_batch_size: 32
  • per_device_eval_batch_size: 8
  • per_gpu_train_batch_size: None
  • per_gpu_eval_batch_size: None
  • gradient_accumulation_steps: 4
  • eval_accumulation_steps: None
  • torch_empty_cache_steps: None
  • learning_rate: 3e-05
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • max_grad_norm: 1.0
  • num_train_epochs: 2
  • max_steps: -1
  • lr_scheduler_type: constant_with_warmup
  • lr_scheduler_kwargs: None
  • warmup_ratio: 0.03
  • warmup_steps: 0
  • log_level: passive
  • log_level_replica: warning
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • save_safetensors: True
  • save_on_each_node: False
  • save_only_model: False
  • restore_callback_states_from_checkpoint: False
  • no_cuda: False
  • use_cpu: False
  • use_mps_device: False
  • seed: 42
  • data_seed: None
  • jit_mode_eval: False
  • bf16: False
  • fp16: True
  • fp16_opt_level: O1
  • half_precision_backend: auto
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • local_rank: 0
  • ddp_backend: None
  • tpu_num_cores: None
  • tpu_metrics_debug: False
  • debug: []
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_prefetch_factor: None
  • past_index: -1
  • disable_tqdm: False
  • remove_unused_columns: True
  • label_names: None
  • load_best_model_at_end: False
  • ignore_data_skip: False
  • fsdp: []
  • fsdp_min_num_params: 0
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • fsdp_transformer_layer_cls_to_wrap: None
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • parallelism_config: None
  • deepspeed: None
  • label_smoothing_factor: 0.0
  • optim: adamw_torch_fused
  • optim_args: None
  • adafactor: False
  • group_by_length: False
  • length_column_name: length
  • project: huggingface
  • trackio_space_id: trackio
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • skip_memory_metrics: True
  • use_legacy_prediction_loop: False
  • push_to_hub: True
  • resume_from_checkpoint: None
  • hub_model_id: dkqjrm/qwen3-embedding-olive-lora
  • hub_strategy: every_save
  • hub_private_repo: None
  • hub_always_push: False
  • hub_revision: None
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • include_inputs_for_metrics: False
  • include_for_metrics: []
  • eval_do_concat_batches: True
  • fp16_backend: auto
  • push_to_hub_model_id: None
  • push_to_hub_organization: None
  • mp_parameters:
  • auto_find_batch_size: False
  • full_determinism: False
  • torchdynamo: None
  • ray_scope: last
  • ddp_timeout: 1800
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • include_tokens_per_second: False
  • include_num_input_tokens_seen: no
  • neftune_noise_alpha: None
  • optim_target_modules: None
  • batch_eval_metrics: False
  • eval_on_start: False
  • use_liger_kernel: False
  • liger_kernel_config: None
  • eval_use_gather_object: False
  • average_tokens_across_devices: True
  • prompts: None
  • batch_sampler: no_duplicates
  • multi_dataset_batch_sampler: proportional
  • router_mapping: {}
  • learning_rate_mapping: {}

Training Logs

Click to expand
Epoch Step Training Loss
0.0049 50 1.5027
0.0098 100 0.8366
0.0147 150 0.6713
0.0195 200 0.5863
0.0244 250 0.53
0.0293 300 0.4562
0.0342 350 0.4061
0.0391 400 0.3899
0.0440 450 0.3417
0.0488 500 0.3367
0.0537 550 0.2948
0.0586 600 0.281
0.0635 650 0.2808
0.0684 700 0.2414
0.0733 750 0.2448
0.0782 800 0.2307
0.0830 850 0.2174
0.0879 900 0.2129
0.0928 950 0.2139
0.0977 1000 0.198
0.1026 1050 0.1797
0.1075 1100 0.1923
0.1124 1150 0.1887
0.1172 1200 0.1789
0.1221 1250 0.1833
0.1270 1300 0.168
0.1319 1350 0.1683
0.1368 1400 0.1536
0.1417 1450 0.1632
0.1465 1500 0.155
0.1514 1550 0.1533
0.1563 1600 0.1442
0.1612 1650 0.1407
0.1661 1700 0.1396
0.1710 1750 0.1388
0.1759 1800 0.1375
0.1807 1850 0.1356
0.1856 1900 0.1335
0.1905 1950 0.1296
0.1954 2000 0.1281
0.2003 2050 0.1379
0.2052 2100 0.1213
0.2101 2150 0.1209
0.2149 2200 0.1142
0.2198 2250 0.1305
0.2247 2300 0.115
0.2296 2350 0.1125
0.2345 2400 0.1159
0.2394 2450 0.1131
0.2442 2500 0.1133
0.2491 2550 0.1126
0.2540 2600 0.109
0.2589 2650 0.1135
0.2638 2700 0.0986
0.2687 2750 0.1127
0.2736 2800 0.114
0.2784 2850 0.1079
0.2833 2900 0.1106
0.2882 2950 0.1112
0.2931 3000 0.1006
0.2980 3050 0.1051
0.3029 3100 0.1105
0.3078 3150 0.1046
0.3126 3200 0.1011
0.3175 3250 0.0962
0.3224 3300 0.1002
0.3273 3350 0.1066
0.3322 3400 0.0907
0.3371 3450 0.0894
0.3419 3500 0.1002
0.3468 3550 0.0894
0.3517 3600 0.0897
0.3566 3650 0.0995
0.3615 3700 0.0949
0.3664 3750 0.0914
0.3713 3800 0.0929
0.3761 3850 0.0841
0.3810 3900 0.0847
0.3859 3950 0.0964
0.3908 4000 0.0937
0.3957 4050 0.0874
0.4006 4100 0.0911
0.4055 4150 0.093
0.4103 4200 0.0867
0.4152 4250 0.0841
0.4201 4300 0.083
0.4250 4350 0.0908
0.4299 4400 0.0829
0.4348 4450 0.0871
0.4396 4500 0.0799
0.4445 4550 0.0777
0.4494 4600 0.0873
0.4543 4650 0.0805
0.4592 4700 0.0851
0.4641 4750 0.0855
0.4690 4800 0.0763
0.4738 4850 0.082
0.4787 4900 0.0699
0.4836 4950 0.0802
0.4885 5000 0.0807
0.4934 5050 0.0746
0.4983 5100 0.0705
0.5032 5150 0.0707
0.5080 5200 0.0827
0.5129 5250 0.0808
0.5178 5300 0.0835
0.5227 5350 0.0782
0.5276 5400 0.0698
0.5325 5450 0.0755
0.5373 5500 0.0743
0.5422 5550 0.0744
0.5471 5600 0.0724
0.5520 5650 0.0781
0.5569 5700 0.0712
0.5618 5750 0.0738
0.5667 5800 0.0692
0.5715 5850 0.0747
0.5764 5900 0.0686
0.5813 5950 0.0761
0.5862 6000 0.0696
0.5911 6050 0.0681
0.5960 6100 0.0714
0.6008 6150 0.0682
0.6057 6200 0.0746
0.6106 6250 0.0638
0.6155 6300 0.0672
0.6204 6350 0.0727
0.6253 6400 0.0711
0.6302 6450 0.0716
0.6350 6500 0.0609
0.6399 6550 0.066
0.6448 6600 0.0709
0.6497 6650 0.0687
0.6546 6700 0.0629
0.6595 6750 0.0693
0.6644 6800 0.0678
0.6692 6850 0.0612
0.6741 6900 0.0653
0.6790 6950 0.0642
0.6839 7000 0.068
0.6888 7050 0.0626
0.6937 7100 0.0623
0.6985 7150 0.0622
0.7034 7200 0.0661
0.7083 7250 0.0597
0.7132 7300 0.0584
0.7181 7350 0.0595
0.7230 7400 0.0647
0.7279 7450 0.0664
0.7327 7500 0.0682
0.7376 7550 0.0621
0.7425 7600 0.0603
0.7474 7650 0.0617
0.7523 7700 0.0554
0.7572 7750 0.056
0.7621 7800 0.0594
0.7669 7850 0.0594
0.7718 7900 0.0618
0.7767 7950 0.0638
0.7816 8000 0.0556
0.7865 8050 0.0608
0.7914 8100 0.0624
0.7962 8150 0.0621
0.8011 8200 0.0653
0.8060 8250 0.0648
0.8109 8300 0.0533
0.8158 8350 0.0584
0.8207 8400 0.0552
0.8256 8450 0.066
0.8304 8500 0.0616
0.8353 8550 0.0648
0.8402 8600 0.0618
0.8451 8650 0.0587
0.8500 8700 0.0616
0.8549 8750 0.0544
0.8598 8800 0.0637
0.8646 8850 0.0621
0.8695 8900 0.0574
0.8744 8950 0.0587
0.8793 9000 0.0606
0.8842 9050 0.0595
0.8891 9100 0.0627
0.8939 9150 0.0564
0.8988 9200 0.0542
0.9037 9250 0.0538
0.9086 9300 0.055
0.9135 9350 0.0562
0.9184 9400 0.0547
0.9233 9450 0.0514
0.9281 9500 0.0574
0.9330 9550 0.0503
0.9379 9600 0.0647
0.9428 9650 0.0554
0.9477 9700 0.0532
0.9526 9750 0.056
0.9575 9800 0.0554
0.9623 9850 0.0535
0.9672 9900 0.0553
0.9721 9950 0.0581
0.9770 10000 0.05
0.9819 10050 0.0571
0.9868 10100 0.0534
0.9916 10150 0.0462
0.9965 10200 0.0508
1.0014 10250 0.0506
1.0063 10300 0.0548
1.0111 10350 0.0476
1.0160 10400 0.0504
1.0209 10450 0.0433
1.0258 10500 0.0499
1.0307 10550 0.0453
1.0356 10600 0.0494
1.0404 10650 0.0456
1.0453 10700 0.0499
1.0502 10750 0.049
1.0551 10800 0.0464
1.0600 10850 0.0483
1.0649 10900 0.0487
1.0698 10950 0.0461
1.0746 11000 0.0433
1.0795 11050 0.0474
1.0844 11100 0.0485
1.0893 11150 0.0462
1.0942 11200 0.0396
1.0991 11250 0.0479
1.1040 11300 0.0471
1.1088 11350 0.0473
1.1137 11400 0.0482
1.1186 11450 0.0412
1.1235 11500 0.0455
1.1284 11550 0.0448
1.1333 11600 0.0531
1.1381 11650 0.0466
1.1430 11700 0.0527
1.1479 11750 0.0465
1.1528 11800 0.0536
1.1577 11850 0.0474
1.1626 11900 0.0515
1.1675 11950 0.0429
1.1723 12000 0.0464
1.1772 12050 0.0463
1.1821 12100 0.0491
1.1870 12150 0.0433
1.1919 12200 0.0466
1.1968 12250 0.0522
1.2017 12300 0.0463
1.2065 12350 0.0528
1.2114 12400 0.0451
1.2163 12450 0.0449
1.2212 12500 0.0475
1.2261 12550 0.0468
1.2310 12600 0.0456
1.2358 12650 0.0411
1.2407 12700 0.0439
1.2456 12750 0.0434
1.2505 12800 0.0475
1.2554 12850 0.0468
1.2603 12900 0.046
1.2652 12950 0.0467
1.2700 13000 0.0429
1.2749 13050 0.0437
1.2798 13100 0.048
1.2847 13150 0.0429
1.2896 13200 0.0507
1.2945 13250 0.0426
1.2994 13300 0.0408
1.3042 13350 0.0468
1.3091 13400 0.0389
1.3140 13450 0.0458
1.3189 13500 0.044
1.3238 13550 0.0417
1.3287 13600 0.0437
1.3335 13650 0.0427
1.3384 13700 0.0444
1.3433 13750 0.0496
1.3482 13800 0.0443
1.3531 13850 0.0421
1.3580 13900 0.0431
1.3629 13950 0.0474
1.3677 14000 0.0423
1.3726 14050 0.0437
1.3775 14100 0.038
1.3824 14150 0.0457
1.3873 14200 0.0459
1.3922 14250 0.0421
1.3970 14300 0.0482
1.4019 14350 0.0496
1.4068 14400 0.0436
1.4117 14450 0.0437
1.4166 14500 0.0463
1.4215 14550 0.04
1.4264 14600 0.046
1.4312 14650 0.0451
1.4361 14700 0.044
1.4410 14750 0.0436
1.4459 14800 0.0411
1.4508 14850 0.0453
1.4557 14900 0.0402
1.4606 14950 0.0437
1.4654 15000 0.0451
1.4703 15050 0.0454
1.4752 15100 0.0433
1.4801 15150 0.0399
1.4850 15200 0.0389
1.4899 15250 0.0451
1.4947 15300 0.0417
1.4996 15350 0.0411
1.5045 15400 0.0415
1.5094 15450 0.044
1.5143 15500 0.045
1.5192 15550 0.0414
1.5241 15600 0.0439
1.5289 15650 0.0381
1.5338 15700 0.0425
1.5387 15750 0.0439
1.5436 15800 0.0405
1.5485 15850 0.0407
1.5534 15900 0.04
1.5583 15950 0.0404
1.5631 16000 0.0392
1.5680 16050 0.0432
1.5729 16100 0.0374
1.5778 16150 0.044
1.5827 16200 0.0429
1.5876 16250 0.0394
1.5924 16300 0.0446
1.5973 16350 0.0389
1.6022 16400 0.0429
1.6071 16450 0.0442
1.6120 16500 0.0394
1.6169 16550 0.0403
1.6218 16600 0.0414
1.6266 16650 0.0386
1.6315 16700 0.0401
1.6364 16750 0.0415
1.6413 16800 0.0427
1.6462 16850 0.0412
1.6511 16900 0.0404
1.6560 16950 0.0402
1.6608 17000 0.0394
1.6657 17050 0.0429
1.6706 17100 0.0452
1.6755 17150 0.0438
1.6804 17200 0.0433
1.6853 17250 0.0393
1.6901 17300 0.0405
1.6950 17350 0.044
1.6999 17400 0.042
1.7048 17450 0.0401
1.7097 17500 0.0417
1.7146 17550 0.0351
1.7195 17600 0.0367
1.7243 17650 0.0436
1.7292 17700 0.0392
1.7341 17750 0.04
1.7390 17800 0.0415
1.7439 17850 0.0418
1.7488 17900 0.0366
1.7537 17950 0.0433
1.7585 18000 0.0391
1.7634 18050 0.0377
1.7683 18100 0.0398
1.7732 18150 0.0396
1.7781 18200 0.0404
1.7830 18250 0.0405
1.7878 18300 0.0381
1.7927 18350 0.04
1.7976 18400 0.0404
1.8025 18450 0.0348
1.8074 18500 0.0397
1.8123 18550 0.042
1.8172 18600 0.0454
1.8220 18650 0.0384
1.8269 18700 0.0387
1.8318 18750 0.042
1.8367 18800 0.0413
1.8416 18850 0.0403
1.8465 18900 0.0417
1.8514 18950 0.0386
1.8562 19000 0.0417
1.8611 19050 0.0396
1.8660 19100 0.039
1.8709 19150 0.0403
1.8758 19200 0.0402
1.8807 19250 0.044
1.8855 19300 0.0413
1.8904 19350 0.0379
1.8953 19400 0.042
1.9002 19450 0.0389
1.9051 19500 0.0399
1.9100 19550 0.0405
1.9149 19600 0.0414
1.9197 19650 0.0406
1.9246 19700 0.037
1.9295 19750 0.0406
1.9344 19800 0.0433
1.9393 19850 0.0357
1.9442 19900 0.038
1.9490 19950 0.0444
1.9539 20000 0.0406
1.9588 20050 0.0343
1.9637 20100 0.0414
1.9686 20150 0.0359
1.9735 20200 0.0421
1.9784 20250 0.0352
1.9832 20300 0.0406
1.9881 20350 0.0403
1.9930 20400 0.0396
1.9979 20450 0.0378

Framework Versions

  • Python: 3.12.12
  • Sentence Transformers: 5.2.1
  • Transformers: 4.57.6
  • PyTorch: 2.10.0+cu128
  • Accelerate: 1.12.0
  • Datasets: 4.3.0
  • Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss 

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
Sentence Similarity
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for dkqjrm/lora_model

Base model

Qwen/Qwen3-0.6B-Base
Finetuned
unsloth/Qwen3-Embedding-0.6B
Finetuned
(2)
this model

Papers for dkqjrm/lora_model