Initial upload

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 768,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

README.md

@@ -0,0 +1,89 @@
+---
+base_model: FacebookAI/roberta-base
+datasets:
+- SynthSTEL/styledistance_training_triplets
+- StyleDistance/synthstel
+language:
+- en
+library_name: sentence-transformers
+license: mit
+pipeline_tag: sentence-similarity
+tags:
+- datadreamer
+- datadreamer-0.35.0
+- synthetic
+- sentence-transformers
+- feature-extraction
+- sentence-similarity
+widget:
+- example_title: Example 1
+  source_sentence: Did you hear about the Wales wing? He'll h8 2 withdraw due 2 injuries
+    from future competitions.
+  sentences:
+  - We're raising funds 2 improve our school's storage facilities and add new playground
+    equipment!
+  - Did you hear about the Wales wing? He'll hate to withdraw due to injuries from
+    future competitions.
+- example_title: Example 2
+  source_sentence: You planned the DesignMeets Decades of Design event; you executed
+    it perfectly.
+  sentences:
+  - We'll find it hard to prove the thief didn't face a real threat!
+  - You orchestrated the DesignMeets Decades of Design gathering; you actualized it
+    flawlessly.
+- example_title: Example 3
+  source_sentence: Did the William Barr maintain a commitment to allow Robert Mueller
+    to finish the inquiry?\
+  sentences:
+  - Will the artist be compiling a music album, or will there be a different focus
+    in the future?
+  - Did William Barr maintain commitment to allow Robert Mueller to finish inquiry?
+---
+
+# Model Card
+
+This repository contains the model introduced in [StyleDistance: Stronger Content-Independent Style Embeddings with Synthetic Parallel Examples](https://huggingface.co/papers/2410.12757).
+
+StyleDistance is a **style embedding model** that aims to embed texts with similar writing styles closely and different styles far apart, regardless of content. You may find this model useful for stylistic analysis of text, clustering, authorship identfication and verification tasks, and automatic style transfer evaluation.
+
+## Training Data and Variants of StyleDistance
+
+StyleDistance was contrastively trained on [SynthSTEL](https://huggingface.co/datasets/StyleDistance/synthstel), a synthetically generated dataset of positive and negative examples of 40 style features being used in text. By utilizing this synthetic dataset, StyleDistance is able to achieve stronger content-independence than other style embeddding models currently available. This particular model was trained using a combination of the synthetic dataset and a [real dataset that makes use of authorship datasets from Reddit to train style embeddings](https://aclanthology.org/2022.repl4nlp-1.26/). For a version that is purely trained on synthetic data, see this other version of [StyleDistance](https://huggingface.co/StyleDistance/styledistance_synthetic_only). 
+
+## Example Usage
+
+```python3
+from sentence_transformers import SentenceTransformer
+from sentence_transformers.util import cos_sim
+
+model = SentenceTransformer('StyleDistance/styledistance') # Load model
+
+input = model.encode("Did you hear about the Wales wing? He'll h8 2 withdraw due 2 injuries from future competitions.")
+others = model.encode(["We're raising funds 2 improve our school's storage facilities and add new playground equipment!", "Did you hear about the Wales wing? He'll hate to withdraw due to injuries from future competitions."])
+print(cos_sim(input, others))
+```
+
+---
+## Citation
+
+```latex
+@misc{patel2025styledistancestrongercontentindependentstyle,
+      title={StyleDistance: Stronger Content-Independent Style Embeddings with Synthetic Parallel Examples}, 
+      author={Ajay Patel and Jiacheng Zhu and Justin Qiu and Zachary Horvitz and Marianna Apidianaki and Kathleen McKeown and Chris Callison-Burch},
+      year={2025},
+      eprint={2410.12757},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2410.12757}, 
+}
+```
+
+---
+## Trained with DataDreamer
+
+This model was trained with a synthetic dataset with [DataDreamer 🤖💤](https://datadreamer.dev). The synthetic dataset card and model card can be found [here](datadreamer.json). The training arguments can be found [here](training_args.json).
+
+---
+#### Funding Acknowledgements
+
+<small> This research is supported in part by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via the HIATUS Program contract #2022-22072200005. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of ODNI, IARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation therein. </small>

classifier.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:09bf2be7c3493f675f9d3c1f954eaae76d27d64d1bd5015c360c119df3ed40e4
+size 11524266

config.json

@@ -0,0 +1,26 @@
+{
+  "architectures": [
+    "RobertaModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "roberta",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.51.3",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 50265
+}

config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "4.1.0",
+    "transformers": "4.51.3",
+    "pytorch": "2.6.0+cu124"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f5151b176d1cce6f0911e572bc56dfa1ef37e2aa00aa51355a5a4ce70a9a981
+size 498604904

modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 512,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50264": {
+      "content": "<mask>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "mask_token": "<mask>",
+  "model_max_length": 512,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "tokenizer_class": "RobertaTokenizer",
+  "trim_offsets": true,
+  "unk_token": "<unk>"
+}