| ---
|
| tags:
|
| - sentence-transformers
|
| - sentence-similarity
|
| - feature-extraction
|
| - dense
|
| - generated_from_trainer
|
| - dataset_size:23522
|
| - loss:SplitHeadContrastiveDistillationLoss
|
| base_model: microsoft/harrier-oss-v1-270m
|
| widget:
|
| - source_sentence: 'Instruct: Retrieve text with a similar pragmatic profile, including
|
| safety, emotion, sentiment, language, and identity-target signals
|
|
|
| Query: "Since women say men only think with their dicks do you think she would
|
| get offended if I asked her to blow my mind." 😭 I hate the people I work with
|
| fucking clowns'
|
| sentences:
|
| - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion,
|
| sentiment, language, and identity-target signals
|
|
|
| Query: /r/ENLIGHTENEDCENTRISM Because someone who wants equality and a nazi are
|
| equally as bad, and homophobes have absolutely *no track record* of not letting
|
| gays keep practicing their ~~comedy~~ life. As opposed to SJWs who have gone into
|
| history responsible for villifying, suppressing and outright killing sexual minorities.
|
| But yeah no, middle ground all the way babyyy. You''re the smartest guy on Reddit!'
|
| - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion,
|
| sentiment, language, and identity-target signals
|
|
|
| Query: they do not care about me or you, they care about what they can take from
|
| you and what they can make you do for them.'
|
| - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion,
|
| sentiment, language, and identity-target signals
|
|
|
| Query: @smkndofpnutdssr @ACLU 70 years ago everyone was brainwashed into being
|
| christian and also had coathanger abortions because it was the Great Depression
|
| and then thousands on women died because they had unsafe abortions 😍'
|
| - source_sentence: 'Instruct: Retrieve text with a similar pragmatic profile, including
|
| safety, emotion, sentiment, language, and identity-target signals
|
|
|
| Query: Real estate is... volatile?
|
|
|
|
|
| Well first off, risk is not volatility, and they''re not even related (yeah, you
|
| have to take uni-level finance courses to learn this...). So don''t conflate
|
| the two terms.
|
|
|
|
|
| Second, any dunce stupid enough to sell real estate for carry forward cap losses
|
| (you also do know that cap losses can only be applied against cap gains, yes?)
|
| is actually going to help bring liquidity into a market, right?'
|
| sentences:
|
| - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion,
|
| sentiment, language, and identity-target signals
|
|
|
| Query: Islam is not a race, it''s a religion, so we can''t be racist'
|
| - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion,
|
| sentiment, language, and identity-target signals
|
|
|
| Query: Really? Who else would ignore 97% of scientists on subject? That is willful
|
| ignorance. They don''t and/or won''t understand climate change. They''re ignorance
|
| is going to kill us all. Including an ignorant president who thinks cc is a Chinese
|
| conspiracy. Talk about arrogance. (and stupidity).'
|
| - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion,
|
| sentiment, language, and identity-target signals
|
|
|
| Query: america is the best country in the world because we are free, and we are
|
| free because we are christian, and we are christian because'
|
| - source_sentence: 'Instruct: Retrieve text with a similar pragmatic profile, including
|
| safety, emotion, sentiment, language, and identity-target signals
|
|
|
| Query: Nasty ass old white men. Send them all to prison right along with RKelly!!'
|
| sentences:
|
| - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion,
|
| sentiment, language, and identity-target signals
|
|
|
| Query: the government should not be in the business of providing services for
|
| those who cannot help themselves'
|
| - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion,
|
| sentiment, language, and identity-target signals
|
|
|
| Query: Isis they are not muslims, they are made of usa stop ur fucking shit 💩'
|
| - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion,
|
| sentiment, language, and identity-target signals
|
|
|
| Query: He hasn''t been effective since 2015. It''s his only option other than
|
| retiring.'
|
| - source_sentence: 'Instruct: Retrieve text with a similar pragmatic profile, including
|
| safety, emotion, sentiment, language, and identity-target signals
|
|
|
| Query: The writer was also upset that she, a mother of 3, didn''t get a chance
|
| to eat that morning. So she wanted to punish the metro worker for eating?????
|
| This lawsuit is the most entitled pile of garbage. She''s accusing the publisher
|
| of attacking her cos she''s a woman of color URL'
|
| sentences:
|
| - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion,
|
| sentiment, language, and identity-target signals
|
|
|
| Query: please shove all the nearest white shit in your mouth and swallow.. & not
|
| your homeboy, either. look past him.'
|
| - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion,
|
| sentiment, language, and identity-target signals
|
|
|
| Query: And not one Representative would accept their version " health care" as
|
| their own.'
|
| - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion,
|
| sentiment, language, and identity-target signals
|
|
|
| Query: This article is complete nonsense and those who buy into it need to get
|
| a real life.'
|
| - source_sentence: 'Instruct: Retrieve text with a similar pragmatic profile, including
|
| safety, emotion, sentiment, language, and identity-target signals
|
|
|
| Query: Everyone in my country has been killing each other for years over religion
|
| and they''re not even different religion just different branches of Christianity
|
| and I quickly realised it''s all pointless'
|
| sentences:
|
| - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion,
|
| sentiment, language, and identity-target signals
|
|
|
| Query: Yea I saw that...had me worried for a sec but I had no trouble at Delta'
|
| - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion,
|
| sentiment, language, and identity-target signals
|
|
|
| Query: Me when my family confronts me about all the queer content on my social
|
| media URL'
|
| - 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion,
|
| sentiment, language, and identity-target signals
|
|
|
| Query: Good to see Tomas Rosicky playing tdae #ARSvQPR'
|
| pipeline_tag: sentence-similarity
|
| library_name: sentence-transformers
|
| ---
|
|
|
| # SentenceTransformer based on microsoft/harrier-oss-v1-270m
|
|
|
| This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [microsoft/harrier-oss-v1-270m](https://huggingface.co/microsoft/harrier-oss-v1-270m). It maps sentences & paragraphs to a 896-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
|
|
|
| ## Model Details
|
|
|
| ### Model Description
|
| - **Model Type:** Sentence Transformer
|
| - **Base model:** [microsoft/harrier-oss-v1-270m](https://huggingface.co/microsoft/harrier-oss-v1-270m) <!-- at revision 31de22b673913c7d658c0f03f792d77c2dcf8ebd -->
|
| - **Maximum Sequence Length:** 512 tokens
|
| - **Output Dimensionality:** 896 dimensions
|
| - **Similarity Function:** Cosine Similarity
|
| <!-- - **Training Dataset:** Unknown -->
|
| <!-- - **Language:** Unknown -->
|
| <!-- - **License:** Unknown -->
|
|
|
| ### Model Sources
|
|
|
| - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
|
| - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
|
| - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
|
|
|
| ### Full Model Architecture
|
|
|
| ```
|
| SentenceTransformer(
|
| (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
|
| (1): Pooling({'word_embedding_dimension': 640, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
|
| (2): Dense({'in_features': 640, 'out_features': 896, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
|
| (3): Normalize()
|
| )
|
| ```
|
|
|
| ## Usage
|
|
|
| ### Direct Usage (Sentence Transformers)
|
|
|
| First install the Sentence Transformers library:
|
|
|
| ```bash
|
| pip install -U sentence-transformers
|
| ```
|
|
|
| Then you can load this model and run inference.
|
| ```python
|
| from sentence_transformers import SentenceTransformer
|
|
|
| # Download from the 🤗 Hub
|
| model = SentenceTransformer("sentence_transformers_model_id")
|
| # Run inference
|
| sentences = [
|
| "Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals\nQuery: Everyone in my country has been killing each other for years over religion and they're not even different religion just different branches of Christianity and I quickly realised it's all pointless",
|
| 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals\nQuery: Me when my family confronts me about all the queer content on my social media URL',
|
| 'Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals\nQuery: Good to see Tomas Rosicky playing tdae #ARSvQPR',
|
| ]
|
| embeddings = model.encode(sentences)
|
| print(embeddings.shape)
|
| # [3, 896]
|
|
|
| # Get the similarity scores for the embeddings
|
| similarities = model.similarity(embeddings, embeddings)
|
| print(similarities)
|
| # tensor([[1.0078, 0.6172, 0.5234],
|
| # [0.6172, 1.0000, 0.5859],
|
| # [0.5234, 0.5859, 1.0000]], dtype=torch.bfloat16)
|
| ```
|
|
|
| <!--
|
| ### Direct Usage (Transformers)
|
|
|
| <details><summary>Click to see the direct usage in Transformers</summary>
|
|
|
| </details>
|
| -->
|
|
|
| <!--
|
| ### Downstream Usage (Sentence Transformers)
|
|
|
| You can finetune this model on your own dataset.
|
|
|
| <details><summary>Click to expand</summary>
|
|
|
| </details>
|
| -->
|
|
|
| <!--
|
| ### Out-of-Scope Use
|
|
|
| *List how the model may foreseeably be misused and address what users ought not to do with the model.*
|
| -->
|
|
|
| <!--
|
| ## Bias, Risks and Limitations
|
|
|
| *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
|
| -->
|
|
|
| <!--
|
| ### Recommendations
|
|
|
| *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
|
| -->
|
|
|
| ## Training Details
|
|
|
| ### Training Dataset
|
|
|
| #### Unnamed Dataset
|
|
|
| * Size: 23,522 training samples
|
| * Columns: <code>sentence_0</code>, <code>sentence_1</code>, and <code>label</code>
|
| * Approximate statistics based on the first 1000 samples:
|
| | | sentence_0 | sentence_1 | label |
|
| |:--------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:--------------------------------------------------------------|
|
| | type | string | string | float |
|
| | details | <ul><li>min: 31 tokens</li><li>mean: 60.22 tokens</li><li>max: 275 tokens</li></ul> | <ul><li>min: 30 tokens</li><li>mean: 59.05 tokens</li><li>max: 262 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.5</li><li>max: 1.0</li></ul> |
|
| * Samples:
|
| | sentence_0 | sentence_1 | label |
|
| |:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------|
|
| | <code>Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals<br>Query: money grubbing filthy kike in panic mode he has to refund shekels</code> | <code>Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals<br>Query: the only ones that have a mental illness are the jews for reading a racial supremacy manifesto that says they are the master race jews always accuse non jews of everything that jews are guilty of</code> | <code>1.0</code> |
|
| | <code>Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals<br>Query: @user SJSHSJ THATS MY JOB BITCH</code> | <code>Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals<br>Query: STOCKS RECORD HIGH URL #MAGA</code> | <code>0.0</code> |
|
| | <code>Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals<br>Query: the best way to do this is to keep them from coming to america, and the best way to keep them from coming to america is to</code> | <code>Instruct: Retrieve text with a similar pragmatic profile, including safety, emotion, sentiment, language, and identity-target signals<br>Query: i have a friend who works in a restaurant and he says that he has never seen a white person working as a busboy or dishwasher</code> | <code>1.0</code> |
|
| * Loss: <code>__main__.SplitHeadContrastiveDistillationLoss</code>
|
|
|
| ### Training Hyperparameters
|
| #### Non-Default Hyperparameters
|
|
|
| - `per_device_train_batch_size`: 16
|
| - `per_device_eval_batch_size`: 16
|
| - `num_train_epochs`: 1
|
| - `multi_dataset_batch_sampler`: round_robin
|
|
|
| #### All Hyperparameters
|
| <details><summary>Click to expand</summary>
|
|
|
| - `overwrite_output_dir`: False
|
| - `do_predict`: False
|
| - `eval_strategy`: no
|
| - `prediction_loss_only`: True
|
| - `per_device_train_batch_size`: 16
|
| - `per_device_eval_batch_size`: 16
|
| - `per_gpu_train_batch_size`: None
|
| - `per_gpu_eval_batch_size`: None
|
| - `gradient_accumulation_steps`: 1
|
| - `eval_accumulation_steps`: None
|
| - `torch_empty_cache_steps`: None
|
| - `learning_rate`: 5e-05
|
| - `weight_decay`: 0.0
|
| - `adam_beta1`: 0.9
|
| - `adam_beta2`: 0.999
|
| - `adam_epsilon`: 1e-08
|
| - `max_grad_norm`: 1
|
| - `num_train_epochs`: 1
|
| - `max_steps`: -1
|
| - `lr_scheduler_type`: linear
|
| - `lr_scheduler_kwargs`: None
|
| - `warmup_ratio`: 0.0
|
| - `warmup_steps`: 0
|
| - `log_level`: passive
|
| - `log_level_replica`: warning
|
| - `log_on_each_node`: True
|
| - `logging_nan_inf_filter`: True
|
| - `save_safetensors`: True
|
| - `save_on_each_node`: False
|
| - `save_only_model`: False
|
| - `restore_callback_states_from_checkpoint`: False
|
| - `no_cuda`: False
|
| - `use_cpu`: False
|
| - `use_mps_device`: False
|
| - `seed`: 42
|
| - `data_seed`: None
|
| - `jit_mode_eval`: False
|
| - `bf16`: False
|
| - `fp16`: False
|
| - `fp16_opt_level`: O1
|
| - `half_precision_backend`: auto
|
| - `bf16_full_eval`: False
|
| - `fp16_full_eval`: False
|
| - `tf32`: None
|
| - `local_rank`: 0
|
| - `ddp_backend`: None
|
| - `tpu_num_cores`: None
|
| - `tpu_metrics_debug`: False
|
| - `debug`: []
|
| - `dataloader_drop_last`: False
|
| - `dataloader_num_workers`: 0
|
| - `dataloader_prefetch_factor`: None
|
| - `past_index`: -1
|
| - `disable_tqdm`: False
|
| - `remove_unused_columns`: True
|
| - `label_names`: None
|
| - `load_best_model_at_end`: False
|
| - `ignore_data_skip`: False
|
| - `fsdp`: []
|
| - `fsdp_min_num_params`: 0
|
| - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
|
| - `fsdp_transformer_layer_cls_to_wrap`: None
|
| - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
|
| - `parallelism_config`: None
|
| - `deepspeed`: None
|
| - `label_smoothing_factor`: 0.0
|
| - `optim`: adamw_torch_fused
|
| - `optim_args`: None
|
| - `adafactor`: False
|
| - `group_by_length`: False
|
| - `length_column_name`: length
|
| - `project`: huggingface
|
| - `trackio_space_id`: trackio
|
| - `ddp_find_unused_parameters`: None
|
| - `ddp_bucket_cap_mb`: None
|
| - `ddp_broadcast_buffers`: False
|
| - `dataloader_pin_memory`: True
|
| - `dataloader_persistent_workers`: False
|
| - `skip_memory_metrics`: True
|
| - `use_legacy_prediction_loop`: False
|
| - `push_to_hub`: False
|
| - `resume_from_checkpoint`: None
|
| - `hub_model_id`: None
|
| - `hub_strategy`: every_save
|
| - `hub_private_repo`: None
|
| - `hub_always_push`: False
|
| - `hub_revision`: None
|
| - `gradient_checkpointing`: False
|
| - `gradient_checkpointing_kwargs`: None
|
| - `include_inputs_for_metrics`: False
|
| - `include_for_metrics`: []
|
| - `eval_do_concat_batches`: True
|
| - `fp16_backend`: auto
|
| - `push_to_hub_model_id`: None
|
| - `push_to_hub_organization`: None
|
| - `mp_parameters`:
|
| - `auto_find_batch_size`: False
|
| - `full_determinism`: False
|
| - `torchdynamo`: None
|
| - `ray_scope`: last
|
| - `ddp_timeout`: 1800
|
| - `torch_compile`: False
|
| - `torch_compile_backend`: None
|
| - `torch_compile_mode`: None
|
| - `include_tokens_per_second`: False
|
| - `include_num_input_tokens_seen`: no
|
| - `neftune_noise_alpha`: None
|
| - `optim_target_modules`: None
|
| - `batch_eval_metrics`: False
|
| - `eval_on_start`: False
|
| - `use_liger_kernel`: False
|
| - `liger_kernel_config`: None
|
| - `eval_use_gather_object`: False
|
| - `average_tokens_across_devices`: True
|
| - `prompts`: None
|
| - `batch_sampler`: batch_sampler
|
| - `multi_dataset_batch_sampler`: round_robin
|
| - `router_mapping`: {}
|
| - `learning_rate_mapping`: {}
|
|
|
| </details>
|
|
|
| ### Training Logs
|
| | Epoch | Step | Training Loss |
|
| |:------:|:----:|:-------------:|
|
| | 0.3399 | 500 | 0.0316 |
|
| | 0.6798 | 1000 | 0.0315 |
|
| | 1.0197 | 1500 | 0.031 |
|
| | 1.3596 | 2000 | 0.0298 |
|
| | 1.6995 | 2500 | 0.0302 |
|
| | 0.3399 | 500 | 0.0288 |
|
| | 0.6798 | 1000 | 0.029 |
|
|
|
|
|
| ### Framework Versions
|
| - Python: 3.14.4
|
| - Sentence Transformers: 5.1.0
|
| - Transformers: 4.57.6
|
| - PyTorch: 2.11.0+cu128
|
| - Accelerate: 1.13.0
|
| - Datasets: 4.8.4
|
| - Tokenizers: 0.22.2
|
|
|
| ## Citation
|
|
|
| ### BibTeX
|
|
|
| #### Sentence Transformers
|
| ```bibtex
|
| @inproceedings{reimers-2019-sentence-bert,
|
| title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
|
| author = "Reimers, Nils and Gurevych, Iryna",
|
| booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
|
| month = "11",
|
| year = "2019",
|
| publisher = "Association for Computational Linguistics",
|
| url = "https://arxiv.org/abs/1908.10084",
|
| }
|
| ```
|
|
|
| <!--
|
| ## Glossary
|
|
|
| *Clearly define terms in order to be accessible across audiences.*
|
| -->
|
|
|
| <!--
|
| ## Model Card Authors
|
|
|
| *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
|
| -->
|
|
|
| <!--
|
| ## Model Card Contact
|
|
|
| *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
|
| --> |
