--- tags: - sentence-transformers - sentence-similarity - feature-extraction - dense - generated_from_trainer - dataset_size:23522 - loss:SplitHeadContrastiveDistillationLoss base_model: microsoft/harrier-oss-v1-270m widget: - source_sentence: 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals Query: "Since women say men only think with their dicks do you think she would get offended if I asked her to blow my mind." 😭 I hate the people I work with fucking clowns' sentences: - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals Query: /r/ENLIGHTENEDCENTRISM Because someone who wants equality and a nazi are equally as bad, and homophobes have absolutely *no track record* of not letting gays keep practicing their ~~comedy~~ life. As opposed to SJWs who have gone into history responsible for villifying, suppressing and outright killing sexual minorities. But yeah no, middle ground all the way babyyy. You''re the smartest guy on Reddit!' - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals Query: they do not care about me or you, they care about what they can take from you and what they can make you do for them.' - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals Query: @smkndofpnutdssr @ACLU 70 years ago everyone was brainwashed into being christian and also had coathanger abortions because it was the Great Depression and then thousands on women died because they had unsafe abortions 😍' - source_sentence: 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals Query: Real estate is... volatile? Well first off, risk is not volatility, and they''re not even related (yeah, you have to take uni-level finance courses to learn this...). So don''t conflate the two terms. Second, any dunce stupid enough to sell real estate for carry forward cap losses (you also do know that cap losses can only be applied against cap gains, yes?) is actually going to help bring liquidity into a market, right?' sentences: - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals Query: Islam is not a race, it''s a religion, so we can''t be racist' - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals Query: Really? Who else would ignore 97% of scientists on subject? That is willful ignorance. They don''t and/or won''t understand climate change. They''re ignorance is going to kill us all. Including an ignorant president who thinks cc is a Chinese conspiracy. Talk about arrogance. (and stupidity).' - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals Query: america is the best country in the world because we are free, and we are free because we are christian, and we are christian because' - source_sentence: 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals Query: Nasty ass old white men. Send them all to prison right along with RKelly!!' sentences: - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals Query: the government should not be in the business of providing services for those who cannot help themselves' - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals Query: Isis they are not muslims, they are made of usa stop ur fucking shit 💩' - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals Query: He hasn''t been effective since 2015. It''s his only option other than retiring.' - source_sentence: 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals Query: The writer was also upset that she, a mother of 3, didn''t get a chance to eat that morning. So she wanted to punish the metro worker for eating????? This lawsuit is the most entitled pile of garbage. She''s accusing the publisher of attacking her cos she''s a woman of color URL' sentences: - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals Query: please shove all the nearest white shit in your mouth and swallow.. & not your homeboy, either. look past him.' - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals Query: And not one Representative would accept their version " health care" as their own.' - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals Query: This article is complete nonsense and those who buy into it need to get a real life.' - source_sentence: 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals Query: Everyone in my country has been killing each other for years over religion and they''re not even different religion just different branches of Christianity and I quickly realised it''s all pointless' sentences: - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals Query: Yea I saw that...had me worried for a sec but I had no trouble at Delta' - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals Query: Me when my family confronts me about all the queer content on my social media URL' - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals Query: Good to see Tomas Rosicky playing tdae #ARSvQPR' pipeline_tag: sentence-similarity library_name: sentence-transformers --- # SentenceTransformer based on microsoft/harrier-oss-v1-270m This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [microsoft/harrier-oss-v1-270m](https://huggingface.co/microsoft/harrier-oss-v1-270m). It maps sentences & paragraphs to a 896-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [microsoft/harrier-oss-v1-270m](https://huggingface.co/microsoft/harrier-oss-v1-270m) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 896 dimensions - **Similarity Function:** Cosine Similarity ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'}) (1): Pooling({'word_embedding_dimension': 640, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True}) (2): Dense({'in_features': 640, 'out_features': 896, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'}) (3): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("sentence_transformers_model_id") # Run inference sentences = [ "Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals\nQuery: Everyone in my country has been killing each other for years over religion and they're not even different religion just different branches of Christianity and I quickly realised it's all pointless", 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals\nQuery: Me when my family confronts me about all the queer content on my social media URL', 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals\nQuery: Good to see Tomas Rosicky playing tdae #ARSvQPR', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 896] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities) # tensor([[1.0078, 0.6172, 0.5234], # [0.6172, 1.0000, 0.5859], # [0.5234, 0.5859, 1.0000]], dtype=torch.bfloat16) ``` ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 23,522 training samples * Columns: sentence_0, sentence_1, and label * Approximate statistics based on the first 1000 samples: | | sentence_0 | sentence_1 | label | |:--------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:--------------------------------------------------------------| | type | string | string | float | | details | | | | * Samples: | sentence_0 | sentence_1 | label | |:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------| | Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals
Query: money grubbing filthy kike in panic mode he has to refund shekels | Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals
Query: the only ones that have a mental illness are the jews for reading a racial supremacy manifesto that says they are the master race jews always accuse non jews of everything that jews are guilty of | 1.0 | | Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals
Query: @user SJSHSJ THATS MY JOB BITCH | Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals
Query: STOCKS RECORD HIGH  URL  #MAGA | 0.0 | | Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals
Query: the best way to do this is to keep them from coming to america, and the best way to keep them from coming to america is to | Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals
Query: i have a friend who works in a restaurant and he says that he has never seen a white person working as a busboy or dishwasher | 1.0 | * Loss: __main__.SplitHeadContrastiveDistillationLoss ### Training Hyperparameters #### Non-Default Hyperparameters - `per_device_train_batch_size`: 16 - `per_device_eval_batch_size`: 16 - `num_train_epochs`: 1 - `multi_dataset_batch_sampler`: round_robin #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: no - `prediction_loss_only`: True - `per_device_train_batch_size`: 16 - `per_device_eval_batch_size`: 16 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 1 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 5e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1 - `num_train_epochs`: 1 - `max_steps`: -1 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: None - `warmup_ratio`: 0.0 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `bf16`: False - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: False - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `parallelism_config`: None - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch_fused - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `project`: huggingface - `trackio_space_id`: trackio - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: None - `hub_always_push`: False - `hub_revision`: None - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `include_for_metrics`: [] - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: no - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `use_liger_kernel`: False - `liger_kernel_config`: None - `eval_use_gather_object`: False - `average_tokens_across_devices`: True - `prompts`: None - `batch_sampler`: batch_sampler - `multi_dataset_batch_sampler`: round_robin - `router_mapping`: {} - `learning_rate_mapping`: {}
### Training Logs | Epoch | Step | Training Loss | |:------:|:----:|:-------------:| | 0.3399 | 500 | 0.0316 | | 0.6798 | 1000 | 0.0315 | | 1.0197 | 1500 | 0.031 | | 1.3596 | 2000 | 0.0298 | | 1.6995 | 2500 | 0.0302 | | 0.3399 | 500 | 0.0288 | | 0.6798 | 1000 | 0.029 | ### Framework Versions - Python: 3.14.4 - Sentence Transformers: 5.1.0 - Transformers: 4.57.6 - PyTorch: 2.11.0+cu128 - Accelerate: 1.13.0 - Datasets: 4.8.4 - Tokenizers: 0.22.2 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ```