psyembedding-roberta-large / README.md

Culture-and-Morality-Lab

Upload 11 files

b50e937 verified 2 months ago

preview code

raw

history blame contribute delete

32.7 kB

metadata

tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - dense
  - generated_from_trainer
  - dataset_size:11180
  - loss:CosineSimilarityLoss
widget:
  - source_sentence: >-
      So when are they leaving? I saw the protest had smaller amounts of
      protestors today as opposed to Friday and Saturday
    sentences:
      - >-
        Hillary's great, but center-leftists seem to do better with younger
        candidates. Bill, Blair, Obama, Trudeau, and now Macron. She'll be 72 in
        2020.
      - >-
        . I felt very sorry for you during your meltdown on  He drove you insane
        but, of course, Piers is a lot smarter than you
      - >-
        As a kid, my friends and I all believed that Gymkata was the most
        violent, bloody movie ever made. I'm not sure who started that rumor. It
        was probably born out of the frustration of 10 year olds who weren't
        allowed to see it for one reason or other. Years after Gymkata was
        released, it became a perennial late night cable movie, and as a result,
        I've been able to make up for lost time. I must have seen scenes from
        this dreadful excuse for a film over a dozen times, and I can always
        spot it from 1-2 seconds of screen time. However, aside from the forced
        coupling of gymnastics and martial arts, the bad dubbing, the stiff
        dialog, and the outrageously difficult story-line, the film has some
        things going for it. With all that's bad about the movie visually, the
        sound is actually pretty entertaining. Never before has a punch or kick
        landed with so little force and so much volume! The canned kung-fu
        sounds are cheeky, but the slowed and pitched-down music, and the nearly
        5 minute slow motion scene are truly weird. The chase through the city
        of demented, blood-thirsty villagers isn't really tense as much as it is
        irritating, and there are enough bad wigs and extras who all but look
        into the camera and wave to make this train-wreck a little fun. Could it
        be headed for cult-classic status? Where is MST3K when we need it?
  - source_sentence: >-
      Seriously, 3 things that really get my blood boiling is hearing about
      child, animal and/or elder abuse. There aint much worse than worthless
      fucks who prey on those in our society who cannot defend themselves.
      Worthless people deserve nothing more than a rope around the neck or life
      in prison at the very least.
    sentences:
      - >-
        As elsewhere, we see polling places and campaign offices getting
        attacked by& checks notes again& oh yeah, also republicans. Yeah, that
        checks out.
      - >-
        Biometric ID System Said to Delay Venezuela Recall By CHRISTOPHER
        TOOTHAKER     CARACAS, Venezuela (AP) -- The high-tech thumbprint
        devices were meant to keep people from voting more than once in the
        recall ballot against President Hugo Chavez. Instead, they often wound
        up working fitfully - even when Chavez himself voted - contributing to
        huge delays in Sunday's historic referendum...
      - >-
        Assuming a republican controlled senate (likely), he will replace Alito
        and Thomas. It won't necessarily move *further* to the right, but we'll
        be stuck with at least 5 conservative justices for the next 40 years.
  - source_sentence: >-
      i should love this movie . the acting is very good and Barbara Stanwyck is
      great but the the movie has always seemed very trite to me . the movie
      makes working class people look low and cheap .the fact that the daughter
      is ashamed of her mother and that the daughter does not rise above it has
      always made me a bit uneasy . Barbara Stanwyck as the mother worships the
      daughter but the daughter forgoes a mothers love to find happiness with
      her well to do fathers family . i wonder how many others who have seen
      this film feel this way about it.again the acting was very very good and
      worth watching . i really don't like the story line . just a personal
      preference .thank you
    sentences:
      - >-
        god ..it takes me back...rolling skating at roller gardens,,,,,you cant
        top old school...the beats back then were so much better...
      - >-
        I'm glad I love my military and the 2nd amendment. #2A #Marines #tlot
        #USA
      - >-
        We definitely need someone better than Trump in 2024, but for now he's
        all we got..
  - source_sentence: >-
      That's just because his right arm is on the inside. Trump knows there's
      nothing he can do to win this round, and he's okay with that. Trump is
      well versed in handshake game strategy, as is Macron clearly.
    sentences:
      - >-
        By Theo Burman - Live News Reporter: 


        Former President Donald Trump and Vice President Kamala Harris are in
        the final sprint to the finishing line in their race to the White House.
        There are 12 days until Election Day and both campaigns are working
        flat-out to win over voters in what is shaping up to be one of the
        closest presidential elections in modern history.


        After events in North Carolina and Georgia earlier this week, Trump is
        continuing his focus on the Sun Belt by heading to Arizona today, while
        Harris is hosting a rally alongside former President Barack Obama and
        Bruce Springsteen in Georgia.


        Read more: [
      - >-
        "&#128514;&#128514;&#128514;&#128514;&#128514;&#128514;&#128514;&#128514;&#128514;&#128514;
        Gay niggas couldn't wait to act like bitches tonight"
      - >-
        When the regressive figure has tremendous power (such as a head of
        state) it's usually not worth risking diplomatic friendship to refuse a
        rather small thing such as wearing a hijab. Le Pen is making wearing a
        hijab a big thing to appeal to both anti-Islam and feminism emotions, in
        other words, risking diplomatic friendship to boost her own popularity.
        Nice move overall.
  - source_sentence: >-
      Unfortunately, the angry masses demand what's not in their best interest
      because of brown people
    sentences:
      - >-
        If Le Pen is perceived to be a US-puppet, wouldn't that rub a lot of
        patriotic/nationalistic voters the wrong way?


        It doesn't seem to be a problem for Trumpists that acknowledge his close
        ties (sic) with Putin.
      - 'I made it 22 years. #metoo'
      - >-
        Secondly, every major support has been leaving the boat during the
        campaign to be in Macron team, thus leaving Hamon alone in an already
        very fragile party.


        Sounds like they made quite a ripple
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
  - pearson_cosine
  - spearman_cosine
model-index:
  - name: SentenceTransformer
    results:
      - task:
          type: semantic-similarity
          name: Semantic Similarity
        dataset:
          name: similarity
          type: similarity
        metrics:
          - type: pearson_cosine
            value: 0.3952284283585713
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.41014481263817126
            name: Spearman Cosine

SentenceTransformer

This is a sentence-transformers model trained. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Maximum Sequence Length: 512 tokens
Output Dimensionality: 1024 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'RobertaModel'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    "Unfortunately, the angry masses demand what's not in their best interest because of brown people",
    'I made it 22 years. #metoo',
    "If Le Pen is perceived to be a US-puppet, wouldn't that rub a lot of patriotic/nationalistic voters the wrong way?\n\nIt doesn't seem to be a problem for Trumpists that acknowledge his close ties (sic) with Putin.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.6501, 0.5940],
#         [0.6501, 1.0000, 0.5664],
#         [0.5940, 0.5664, 1.0000]])

Evaluation

Metrics

Semantic Similarity

Dataset: similarity
Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.3952
spearman_cosine	0.4101

Training Details

Training Dataset

Unnamed Dataset

Size: 11,180 training samples
Columns: sentence_0, sentence_1, and label
Approximate statistics based on the first 1000 samples:
sentence_0 sentence_1 label
type string string float
details
min: 5 tokens
mean: 102.41 tokens
max: 512 tokens

min: 5 tokens
mean: 111.27 tokens
max: 512 tokens

min: 0.0
mean: 0.53
max: 1.0

	sentence_0	sentence_1	label
type	string	string	float
details	min: 5 tokens mean: 102.41 tokens max: 512 tokens	min: 5 tokens mean: 111.27 tokens max: 512 tokens	min: 0.0 mean: 0.53 max: 1.0

Samples:

sentence_0	sentence_1	label
`We love peace, but not peace at any price.`	`That's totally not corrupt whatsoever. Also why the hell is a state attorney general meddling in federal government?`	`0.7071067811865475`
Am not from America, I usually watch this show on AXN channel, I don't know why this respected channel air such sucking program in prime time slot. Creation of Hollywood's Money Bank Jerry Bruckheimer, this time he is spending a big load of cash in the small screen. In each episode a bunch of peoples having two team members travels from on country to another for a great sum of money; where the camera crews shoot their travels. I don't know who the hell gave this stupid idea for the show. It has nothing to watch for, in all episodes we see people ran like beggars, some times shouting, crying, beeping, jerky camera works..huh it's harmful to both eyes and ears. The most disgusting part in the race is the viewers finally knows each of the team members can't enjoy their race/traveling experience. Even though, to add up the ratings the producers came up with the ideas of including Gays in one shows, sucking American reality show.It's nothing to watch for, better switch to another channels.T...	Background: Last year my [41F] brother, Gabe [36M] came to visit around my bday. There is a nice restaurant my family goes to for special occasions, and since Gabe is a chef, I was excited to take him. I made a rez for me, my SO, my kids [23NB, 21F], Gabe, and my sister, Ronnie [35F]. We had a great time. It was "adults only," so my nephews [15, 13] did not come. Since I invited them, we paid; the bill was about $400. Gabe came to visit again in Sept, only stopping for a few days (arrived Sun eve, leaving early Wed am), on his way back home across the country. Asking if he wanted to do anything while in town, he said he'd like to go to that restaurant again. When we saw Ronnie (Sunday), I told her we were going "and you are coming with us." Monday, I took the day off to hang out with Gabe, my sis had to work, but she didn't come over when she got off at 7pm. Tuesday she came over with my nephews around 11am, with dinner rez for 6 ppl (same as last time) at 8pm. We hung out and as th...	`0.3535533905932737`
I (M29) am trans. My girlfriend (F28, GF) is totally cool with it, always has been, we've been dating since college, 8 years in March. GF's dad was abusive, so she left home at 18 and had to leave her baby sister behind. 2015, we're 24/23, in grad school, living together. GF gets some news: her dad died and, long story short, nobody can take her sister in. We hire a lawyer to try for custody. I quit school to work fulltime so we can afford it. It takes a lot of time and work, but we get to take her home. Fast forward to now. Kid (12, S) has school in person on Tu/Th, virtual learning the rest. Friday the 11th, while S was out walking the dog, I grabbed the hamper out of their room to do the laundry. The pocket of the hoodie they just wore to school was bunched up weird, so I checked it and pulled out a binder. For those who don't know, a binder is usually used by trans people to flatten their chests so they can pass easier. The only other reason I could think of for someone to o...	`Scores plan to leave Mormon church over its policy on same-sex couples - Gay Star News`	`0.4082482904638631`

Loss: CosineSimilarityLoss with these parameters:

{
    "loss_fct": "torch.nn.modules.loss.MSELoss"
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 32
per_device_eval_batch_size: 32
fp16: True
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 32
per_device_eval_batch_size: 32
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 3
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: True
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	similarity_spearman_cosine
0.0286	10	-	0.0535
0.0571	20	-	0.0570
0.0857	30	-	0.0681
0.1143	40	-	0.0739
0.1429	50	-	0.0572
0.1714	60	-	0.0250
0.2	70	-	0.0230
0.2286	80	-	0.0726
0.2571	90	-	0.0548
0.2857	100	-	0.0451
0.3143	110	-	0.0067
0.3429	120	-	0.0425
0.3714	130	-	0.0920
0.4	140	-	0.0823
0.4286	150	-	0.1165
0.4571	160	-	0.1405
0.4857	170	-	0.1661
0.5143	180	-	0.1657
0.5429	190	-	0.1832
0.5714	200	-	0.0056
0.6	210	-	0.1209
0.6286	220	-	0.1280
0.6571	230	-	0.1902
0.6857	240	-	0.2111
0.7143	250	-	0.2717
0.7429	260	-	0.2716
0.7714	270	-	0.2629
0.8	280	-	0.2171
0.8286	290	-	0.2742
0.8571	300	-	0.2913
0.8857	310	-	0.2813
0.9143	320	-	0.2863
0.9429	330	-	0.2918
0.9714	340	-	0.2951
1.0	350	-	0.3198
1.0286	360	-	0.3145
1.0571	370	-	0.3148
1.0857	380	-	0.2907
1.1143	390	-	0.3267
1.1429	400	-	0.3246
1.1714	410	-	0.3351
1.2	420	-	0.3463
1.2286	430	-	0.3531
1.2571	440	-	0.3398
1.2857	450	-	0.3169
1.3143	460	-	0.3304
1.3429	470	-	0.3315
1.3714	480	-	0.3684
1.4	490	-	0.3499
1.4286	500	0.1429	0.3438
1.4571	510	-	0.3362
1.4857	520	-	0.3130
1.5143	530	-	0.3445
1.5429	540	-	0.3464
1.5714	550	-	0.3499
1.6	560	-	0.3626
1.6286	570	-	0.3743
1.6571	580	-	0.3714
1.6857	590	-	0.3774
1.7143	600	-	0.3624
1.7429	610	-	0.3861
1.7714	620	-	0.3925
1.8	630	-	0.3763
1.8286	640	-	0.3906
1.8571	650	-	0.4034
1.8857	660	-	0.3887
1.9143	670	-	0.3970
1.9429	680	-	0.3787
1.9714	690	-	0.3958
2.0	700	-	0.3812
2.0286	710	-	0.3951
2.0571	720	-	0.4066
2.0857	730	-	0.4030
2.1143	740	-	0.4029
2.1429	750	-	0.3899
2.1714	760	-	0.3898
2.2	770	-	0.3987
2.2286	780	-	0.4007
2.2571	790	-	0.4040
2.2857	800	-	0.4101

Framework Versions

Python: 3.11.9
Sentence Transformers: 5.1.0
Transformers: 4.53.3
PyTorch: 2.5.1
Accelerate: 1.10.0
Datasets: 2.14.4
Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}