SentenceTransformer based on unsloth/Qwen3-Embedding-0.6B

This is a sentence-transformers model finetuned from unsloth/Qwen3-Embedding-0.6B on the augmented-olive-product-phonetic-wo-negatives dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: unsloth/Qwen3-Embedding-0.6B
Maximum Sequence Length: 512 tokens
Output Dimensionality: 1024 dimensions
Similarity Function: Cosine Similarity
Training Dataset:
- augmented-olive-product-phonetic-wo-negatives

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'PeftModelForFeatureExtraction'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("dkqjrm/qwen06-embedding-augmented-olive-phonetic-wo-negative-lora")
# Run inference
sentences = [
    '벤시몽 BSM 로고 오가닉 코튼 양말 1개',
    '棉袜子',
    '유분 순삭 드라이 샴푸',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.4293, -0.0163],
#         [ 0.4293,  1.0000, -0.0177],
#         [-0.0163, -0.0177,  1.0000]])

Training Details

Training Dataset

augmented-olive-product-phonetic-wo-negatives

Dataset: augmented-olive-product-phonetic-wo-negatives at e21876d
Size: 2,189,797 training samples
Columns: anchor and positive
Approximate statistics based on the first 1000 samples:
anchor positive
type string string
details
min: 6 tokens
mean: 27.13 tokens
max: 64 tokens

min: 2 tokens
mean: 7.21 tokens
max: 44 tokens

	anchor	positive
type	string	string
details	min: 6 tokens mean: 27.13 tokens max: 64 tokens	min: 2 tokens mean: 7.21 tokens max: 44 tokens

Samples:

anchor	positive
`엘리자베스아덴 그린티 센트스프레이 100ml`	`Green Tea perfume.`
`164 루테인지아잔틴 메모리케어 GR 30캡슐 (30일분)`	`164 루테인지아잔틴 메모리케어 GR 30캡슐 30일분`
`[지성두피&피지 제거] 클로란 네틀 노세범 드라이 샴푸 150ml`	`Dry shampoo`

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false
}

Evaluation Dataset

augmented-olive-product-phonetic-wo-negatives

Dataset: augmented-olive-product-phonetic-wo-negatives at e21876d
Size: 115,252 evaluation samples
Columns: anchor and positive
Approximate statistics based on the first 1000 samples:
anchor positive
type string string
details
min: 6 tokens
mean: 26.68 tokens
max: 63 tokens

min: 2 tokens
mean: 7.45 tokens
max: 47 tokens

	anchor	positive
type	string	string
details	min: 6 tokens mean: 26.68 tokens max: 63 tokens	min: 2 tokens mean: 7.45 tokens max: 47 tokens

Samples:

anchor	positive
`아임쏘리포마이스킨 pH5.5 모이스처 젤리마스크`	`ウルトゥルージェリーマスク`
`블랙몬스터 다운펌 프로 키트`	`블랙몬스터 다운펌 프로`
`그린벨 NEW미세모용족집게(블랙)`	`黑色镊子`

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
gradient_accumulation_steps: 32
learning_rate: 3e-05
num_train_epochs: 2
lr_scheduler_type: cosine
warmup_ratio: 0.1
fp16: True
push_to_hub: True
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 8
per_device_eval_batch_size: 8
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 32
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 3e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 2
max_steps: -1
lr_scheduler_type: cosine
lr_scheduler_kwargs: None
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
bf16: False
fp16: True
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: True
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
prompts: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss	Validation Loss
0.0058	50	0.9157	-
0.0117	100	0.5724	-
0.0175	150	0.4076	-
0.0234	200	0.3557	-
0.0292	250	0.3075	-
0.0351	300	0.2783	-
0.0409	350	0.2603	-
0.0468	400	0.2363	-
0.0526	450	0.2278	-
0.0585	500	0.2194	-
0.0643	550	0.2042	-
0.0701	600	0.1857	-
0.0760	650	0.177	-
0.0818	700	0.1667	-
0.0877	750	0.1665	-
0.0935	800	0.1466	-
0.0994	850	0.1519	-
0.1052	900	0.1428	-
0.1111	950	0.1269	-
0.1169	1000	0.1311	-
0.1228	1050	0.1244	-
0.1286	1100	0.1147	-
0.1344	1150	0.1146	-
0.1403	1200	0.1148	-
0.1461	1250	0.1029	-
0.1520	1300	0.0978	-
0.1578	1350	0.0976	-
0.1637	1400	0.0937	-
0.1695	1450	0.0922	-
0.1754	1500	0.0926	-
0.1812	1550	0.0937	-
0.1870	1600	0.0855	-
0.1929	1650	0.083	-
0.1987	1700	0.0813	-
0.2046	1750	0.0837	-
0.2104	1800	0.0793	-
0.2163	1850	0.0764	-
0.2221	1900	0.0739	-
0.2280	1950	0.0721	-
0.2338	2000	0.0764	-
0.2397	2050	0.0689	-
0.2455	2100	0.0671	-
0.2513	2150	0.0713	-
0.2572	2200	0.0652	-
0.2630	2250	0.0666	-
0.2689	2300	0.0695	-
0.2747	2350	0.0656	-
0.2806	2400	0.0678	-
0.2864	2450	0.0644	-
0.2923	2500	0.0551	-
0.2981	2550	0.0571	-
0.3040	2600	0.0557	-
0.3098	2650	0.0503	-
0.3156	2700	0.0549	-
0.3215	2750	0.053	-
0.3273	2800	0.053	-
0.3332	2850	0.0511	-
0.3390	2900	0.0528	-
0.3449	2950	0.0512	-
0.3507	3000	0.0524	0.0516
0.3566	3050	0.0512	-
0.3624	3100	0.0491	-
0.3683	3150	0.0479	-
0.3741	3200	0.0464	-
0.3799	3250	0.0483	-
0.3858	3300	0.0533	-
0.3916	3350	0.0495	-
0.3975	3400	0.0433	-
0.4033	3450	0.0489	-
0.4092	3500	0.0469	-
0.4150	3550	0.0447	-
0.4209	3600	0.0479	-
0.4267	3650	0.0444	-
0.4326	3700	0.0436	-
0.4384	3750	0.0426	-
0.4442	3800	0.0443	-
0.4501	3850	0.0412	-
0.4559	3900	0.0411	-
0.4618	3950	0.0432	-
0.4676	4000	0.0471	-
0.4735	4050	0.0392	-
0.4793	4100	0.0443	-
0.4852	4150	0.0376	-
0.4910	4200	0.0434	-
0.4968	4250	0.0405	-
0.5027	4300	0.0389	-
0.5085	4350	0.0404	-
0.5144	4400	0.0389	-
0.5202	4450	0.0425	-
0.5261	4500	0.0371	-
0.5319	4550	0.0376	-
0.5378	4600	0.0355	-
0.5436	4650	0.04	-
0.5495	4700	0.0358	-
0.5553	4750	0.0365	-
0.5611	4800	0.0383	-
0.5670	4850	0.0345	-
0.5728	4900	0.0382	-
0.5787	4950	0.0379	-
0.5845	5000	0.0377	-
0.5904	5050	0.034	-
0.5962	5100	0.0363	-
0.6021	5150	0.0347	-
0.6079	5200	0.0343	-
0.6138	5250	0.0339	-
0.6196	5300	0.0358	-
0.6254	5350	0.0351	-
0.6313	5400	0.0327	-
0.6371	5450	0.0361	-
0.6430	5500	0.0346	-
0.6488	5550	0.0343	-
0.6547	5600	0.0329	-
0.6605	5650	0.0295	-
0.6664	5700	0.033	-
0.6722	5750	0.033	-
0.6781	5800	0.033	-
0.6839	5850	0.0313	-
0.6897	5900	0.0305	-
0.6956	5950	0.0329	-
0.7014	6000	0.0309	0.0327
0.7073	6050	0.0331	-
0.7131	6100	0.0304	-
0.7190	6150	0.0304	-
0.7248	6200	0.0338	-
0.7307	6250	0.0344	-
0.7365	6300	0.0312	-
0.7424	6350	0.03	-
0.7482	6400	0.0327	-
0.7540	6450	0.0323	-
0.7599	6500	0.0294	-
0.7657	6550	0.0304	-
0.7716	6600	0.0261	-
0.7774	6650	0.0295	-
0.7833	6700	0.0281	-
0.7891	6750	0.0293	-
0.7950	6800	0.0283	-
0.8008	6850	0.0293	-
0.8066	6900	0.0293	-
0.8125	6950	0.0305	-
0.8183	7000	0.029	-
0.8242	7050	0.0309	-
0.8300	7100	0.0302	-
0.8359	7150	0.0289	-
0.8417	7200	0.0286	-
0.8476	7250	0.0273	-
0.8534	7300	0.0287	-
0.8593	7350	0.0283	-
0.8651	7400	0.0259	-
0.8709	7450	0.0273	-
0.8768	7500	0.0288	-
0.8826	7550	0.0262	-
0.8885	7600	0.0292	-
0.8943	7650	0.0273	-
0.9002	7700	0.0257	-
0.9060	7750	0.0285	-
0.9119	7800	0.0276	-
0.9177	7850	0.0242	-
0.9236	7900	0.0261	-
0.9294	7950	0.0254	-
0.9352	8000	0.0281	-
0.9411	8050	0.0272	-
0.9469	8100	0.0281	-
0.9528	8150	0.0275	-
0.9586	8200	0.0258	-
0.9645	8250	0.0276	-
0.9703	8300	0.0267	-
0.9762	8350	0.0251	-
0.9820	8400	0.0232	-
0.9879	8450	0.0244	-
0.9937	8500	0.027	-
0.9995	8550	0.0244	-
1.0054	8600	0.0231	-
1.0112	8650	0.0206	-
1.0171	8700	0.0246	-
1.0229	8750	0.0219	-
1.0288	8800	0.0225	-
1.0346	8850	0.0219	-
1.0404	8900	0.021	-
1.0463	8950	0.0221	-
1.0521	9000	0.0209	0.0242
1.0580	9050	0.0237	-
1.0638	9100	0.0223	-
1.0697	9150	0.0223	-
1.0755	9200	0.0241	-
1.0814	9250	0.023	-
1.0872	9300	0.022	-
1.0931	9350	0.0225	-
1.0989	9400	0.0224	-
1.1047	9450	0.0186	-
1.1106	9500	0.0249	-
1.1164	9550	0.0226	-
1.1223	9600	0.0219	-
1.1281	9650	0.0227	-
1.1340	9700	0.0204	-
1.1398	9750	0.0211	-
1.1457	9800	0.0224	-
1.1515	9850	0.0227	-
1.1574	9900	0.0213	-
1.1632	9950	0.0214	-
1.1690	10000	0.0201	-
1.1749	10050	0.0223	-
1.1807	10100	0.0201	-
1.1866	10150	0.0187	-
1.1924	10200	0.0209	-
1.1983	10250	0.0223	-
1.2041	10300	0.0193	-
1.2100	10350	0.0205	-
1.2158	10400	0.0202	-
1.2217	10450	0.0214	-
1.2275	10500	0.019	-
1.2333	10550	0.0203	-
1.2392	10600	0.0209	-
1.2450	10650	0.0201	-
1.2509	10700	0.0195	-
1.2567	10750	0.0212	-
1.2626	10800	0.0211	-
1.2684	10850	0.0206	-
1.2743	10900	0.0184	-
1.2801	10950	0.0198	-
1.2860	11000	0.0203	-
1.2918	11050	0.0196	-
1.2976	11100	0.0216	-
1.3035	11150	0.0173	-
1.3093	11200	0.0184	-
1.3152	11250	0.0207	-
1.3210	11300	0.0187	-
1.3269	11350	0.0192	-
1.3327	11400	0.0198	-
1.3386	11450	0.0186	-
1.3444	11500	0.0179	-
1.3502	11550	0.0177	-
1.3561	11600	0.0176	-
1.3619	11650	0.0206	-
1.3678	11700	0.0194	-
1.3736	11750	0.018	-
1.3795	11800	0.0185	-
1.3853	11850	0.0184	-
1.3912	11900	0.0197	-
1.3970	11950	0.018	-
1.4029	12000	0.0165	0.0208
1.4087	12050	0.0182	-
1.4145	12100	0.0175	-
1.4204	12150	0.0173	-
1.4262	12200	0.0203	-
1.4321	12250	0.0202	-
1.4379	12300	0.0187	-
1.4438	12350	0.019	-
1.4496	12400	0.0184	-
1.4555	12450	0.0176	-
1.4613	12500	0.0174	-
1.4672	12550	0.0164	-
1.4730	12600	0.0185	-
1.4788	12650	0.0169	-
1.4847	12700	0.0184	-
1.4905	12750	0.0173	-
1.4964	12800	0.0176	-
1.5022	12850	0.0203	-
1.5081	12900	0.0198	-
1.5139	12950	0.0165	-
1.5198	13000	0.0189	-
1.5256	13050	0.0196	-
1.5315	13100	0.0182	-
1.5373	13150	0.0187	-
1.5431	13200	0.018	-
1.5490	13250	0.0186	-
1.5548	13300	0.0182	-
1.5607	13350	0.0184	-
1.5665	13400	0.0183	-
1.5724	13450	0.0202	-
1.5782	13500	0.0202	-
1.5841	13550	0.0179	-
1.5899	13600	0.0197	-
1.5958	13650	0.0192	-
1.6016	13700	0.0193	-
1.6074	13750	0.0159	-
1.6133	13800	0.0191	-
1.6191	13850	0.0181	-
1.6250	13900	0.0189	-
1.6308	13950	0.0194	-
1.6367	14000	0.0191	-
1.6425	14050	0.0165	-
1.6484	14100	0.0167	-
1.6542	14150	0.0179	-
1.6600	14200	0.0167	-
1.6659	14250	0.0181	-
1.6717	14300	0.0174	-
1.6776	14350	0.0163	-
1.6834	14400	0.0173	-
1.6893	14450	0.0164	-
1.6951	14500	0.0175	-
1.7010	14550	0.0195	-
1.7068	14600	0.0169	-
1.7127	14650	0.0177	-
1.7185	14700	0.0171	-
1.7243	14750	0.0179	-
1.7302	14800	0.0162	-
1.7360	14850	0.0167	-
1.7419	14900	0.0178	-
1.7477	14950	0.0179	-
1.7536	15000	0.0187	0.0189
1.7594	15050	0.0177	-
1.7653	15100	0.0171	-
1.7711	15150	0.0172	-
1.7770	15200	0.0181	-
1.7828	15250	0.0176	-
1.7886	15300	0.0175	-
1.7945	15350	0.0175	-
1.8003	15400	0.0158	-
1.8062	15450	0.0154	-
1.8120	15500	0.0186	-
1.8179	15550	0.0171	-
1.8237	15600	0.017	-
1.8296	15650	0.0164	-
1.8354	15700	0.0165	-
1.8413	15750	0.0169	-
1.8471	15800	0.0174	-
1.8529	15850	0.0174	-
1.8588	15900	0.017	-
1.8646	15950	0.0166	-
1.8705	16000	0.0157	-
1.8763	16050	0.0173	-
1.8822	16100	0.0175	-
1.8880	16150	0.0177	-
1.8939	16200	0.0179	-
1.8997	16250	0.0175	-
1.9056	16300	0.0183	-
1.9114	16350	0.0175	-
1.9172	16400	0.0167	-
1.9231	16450	0.0185	-
1.9289	16500	0.0162	-
1.9348	16550	0.0161	-
1.9406	16600	0.0188	-
1.9465	16650	0.0172	-
1.9523	16700	0.0186	-
1.9582	16750	0.0178	-
1.9640	16800	0.0179	-
1.9698	16850	0.019	-
1.9757	16900	0.0188	-
1.9815	16950	0.0158	-
1.9874	17000	0.016	-
1.9932	17050	0.0184	-
1.9991	17100	0.0194	-

Framework Versions

Python: 3.12.12
Sentence Transformers: 5.2.1
Transformers: 4.57.6
PyTorch: 2.10.0+cu128
Accelerate: 1.12.0
Datasets: 4.3.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

Downloads last month: -; Downloads are not tracked for this model. How to track

Model tree for dkqjrm/qwen06-embedding-augmented-olive-phonetic-wo-negative-lora

Base model

Qwen/Qwen3-0.6B-Base

Finetuned

unsloth/Qwen3-Embedding-0.6B

Finetuned

(7)

this model

Papers for dkqjrm/qwen06-embedding-augmented-olive-phonetic-wo-negative-lora

Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

Paper • 1908.10084 • Published Aug 27, 2019 • 13

Efficient Natural Language Response Suggestion for Smart Reply

Paper • 1705.00652 • Published May 1, 2017