Big5-Personality-MPNet: Personality-Aware Embeddings for Author Similarity Retrieval

This is a sentence-transformers model finetuned from sentence-transformers/all-mpnet-base-v2 on FIPI (Five-Item Personality Inventory) self-report Big Five scores from the psychometric NLP testbed of Abbasi et al. (2021). It maps free-text writing to a 768-dimensional dense vector space where texts written by authors with similar Big Five personality profiles are placed close together, enabling personality-conditioned text retrieval (given a text, retrieve other texts whose authors have similar personality traits).

On strictly thresholded held-out FIPI triplets (positive Manhattan distance ≤ 2, negative Manhattan distance ≥ 20 on the 5-trait FIPI vector), the model achieves 75.95% triplet accuracy, outperforming the pre-trained MPNet baseline (68.10%) as well as OpenAI's text-embedding-3-large (71.67%) and text-embedding-3-small (71.19%) on the same test set.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: sentence-transformers/all-mpnet-base-v2
  • Maximum Sequence Length: 384 tokens
  • Output Dimensionality: 768 dimensions
  • Similarity Function: Cosine Similarity
  • Training Objective: Personality-distance-based triplet learning (Manhattan distance over raw FIPI vectors)
  • Training Data: ~6,760 participants from Abbasi et al. (2021) → ~27K mined triplets
  • Language: English

Model Sources

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 384, 'do_lower_case': False, 'architecture': 'MPNetModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference:

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("NextGen-Institute/big5-personality-mpnet")

# Run inference on three texts written by three different authors
sentences = [
    "I always plan my doctor visits well in advance and write down every question I want to ask. I research my conditions thoroughly and verify what the doctor tells me against multiple sources before making any decision about my treatment.",
    "I make detailed lists for every appointment and follow my doctor's instructions exactly. I keep a folder with all my test results and read up on anything new before my visits.",
    "Honestly I hate doctors and try to avoid them. When I do go I just nod and leave as fast as I can. I figure stuff out on my own usually."
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# Authors 1 and 2 should be more similar to each other (both methodical,
# high-conscientiousness writing style) than either is to author 3.

Evaluation

Triplet Accuracy

We evaluate triplet accuracy on a held-out set of 1,680 strictly thresholded FIPI triplets (positive Manhattan distance ≤ 2, negative Manhattan distance ≥ 20 over the 5-dimensional FIPI vector). A triplet is correctly ranked when cos_sim(anchor, positive) > cos_sim(anchor, negative). Random guessing yields ~50% accuracy.

Model Dimensions Triplet Accuracy
sentence-transformers/all-mpnet-base-v2 (baseline) 768 68.10%
OpenAI text-embedding-3-small 1,536 71.19%
OpenAI text-embedding-3-large 3,072 71.67%
Big5-Personality-MPNet (this model) 768 75.95%

Cosine Similarity Statistics

Model Sim(A, P) Mean ± SD Sim(A, N) Mean ± SD
Baseline MPNet 0.582 ± 0.122 0.481 ± 0.199
OpenAI Small 0.602 ± 0.087 0.534 ± 0.114
OpenAI Large 0.582 ± 0.089 0.505 ± 0.120
This model 0.909 ± 0.042 0.850 ± 0.088

The fine-tuned model produces tighter (lower-variance) similarity distributions. While the absolute margin between mean similarities is small (0.060), reduced variance produces less overlap between the anchor-positive and anchor-negative distributions, which translates into higher triplet accuracy.

Intended Use and Limitations

Intended Use

This model is intended for research use in:

  • Personality-conditioned dense retrieval (find texts by authors with similar Big Five profiles)
  • Computational social science studies of author-level latent variables
  • Health communication research where personality may correlate with how patients describe symptoms, express anxiety, or engage with health information

Out-of-Scope Use

This model should not be used for:

  • Inferring personality traits of individuals who have not consented to such inference
  • Hiring, lending, insurance, or any high-stakes decision-making about individuals
  • Targeted manipulation, advertising, or political messaging based on inferred personality
  • Diagnostic or clinical decision-making

Known Limitations

  • Single-item-per-trait labels. Training labels come from FIPI, which uses a single Likert item per Big Five trait. Brief inventories have lower reliability than multi-item instruments (e.g., BFI-2, NEO-PI-R), placing an upper bound on achievable accuracy.
  • Single-domain training text. All training essays come from four health-domain prompts (subjective health literacy, trust in physicians, anxiety visiting doctors, health numeracy). Personality cues learned by the model may be specific to how individuals write about health and may not transfer to other writing contexts (social media, workplace, creative writing).
  • Demographic skew. Training participants were recruited via Amazon Mechanical Turk and Qualtrics, which skew younger, more educated, and more internet-literate than the general population.
  • English only.
  • Evaluation circularity. Triplet accuracy is computed against the same Manhattan-distance labeling rule used to construct the triplets. The model is shown to recover this rule from text better than baselines, but external personality-validity evidence (e.g., predicting held-out FIPI scores via linear probe) is not provided here.

Training Details

Training Dataset

Free-text health-domain responses paired with FIPI Big Five self-report scores from the psychometric NLP testbed of Abbasi et al. (2021). Each participant contributed four open-ended responses to prompts about subjective health literacy, trust in physicians, anxiety visiting doctors, and health numeracy. The four responses are shuffled and concatenated with double-newline separators to form one "mini-essay" per participant.

  • Total participants: 8,502 (after filtering for valid FIPI + non-empty text: ~8,450)
  • Split: 80/10/10 at the participant level (seed=42) → 6,760 train / 845 val / 845 test
  • Triplet mining: Manhattan distance over raw FIPI vectors (range 0–30)
  • Training thresholds: positive distance ≤ 3, negative distance ≥ 15
  • Evaluation thresholds: positive distance ≤ 2, negative distance ≥ 20
  • Pairing constraints: ≤ 2 positives per anchor, ≤ 2 negatives per positive
  • Training set size: ~26,960 triplets

Note: The raw essay text from Abbasi et al. (2021) is not redistributed with this model. Researchers wishing to reproduce training should obtain the original testbed directly.

Loss

MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false,
    "directions": ["query_to_doc"],
    "partition_mode": "joint",
    "hardness_mode": null,
    "hardness_strength": 0.0
}

Training Hyperparameters

Non-Default Hyperparameters

  • fp16: True
  • batch_sampler: no_duplicates
  • multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand
  • per_device_train_batch_size: 8
  • num_train_epochs: 3
  • max_steps: -1
  • learning_rate: 5e-05
  • lr_scheduler_type: linear
  • lr_scheduler_kwargs: None
  • warmup_steps: 0
  • optim: adamw_torch_fused
  • optim_args: None
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • optim_target_modules: None
  • gradient_accumulation_steps: 1
  • average_tokens_across_devices: True
  • max_grad_norm: 1
  • label_smoothing_factor: 0.0
  • bf16: False
  • fp16: True
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • use_liger_kernel: False
  • liger_kernel_config: None
  • use_cache: False
  • neftune_noise_alpha: None
  • torch_empty_cache_steps: None
  • auto_find_batch_size: False
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • include_num_input_tokens_seen: no
  • log_level: passive
  • log_level_replica: warning
  • disable_tqdm: False
  • project: huggingface
  • trackio_space_id: trackio
  • eval_strategy: no
  • per_device_eval_batch_size: 8
  • prediction_loss_only: True
  • eval_on_start: False
  • eval_do_concat_batches: True
  • eval_use_gather_object: False
  • eval_accumulation_steps: None
  • include_for_metrics: []
  • batch_eval_metrics: False
  • save_only_model: False
  • save_on_each_node: False
  • enable_jit_checkpoint: False
  • push_to_hub: False
  • hub_private_repo: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_always_push: False
  • hub_revision: None
  • load_best_model_at_end: False
  • ignore_data_skip: False
  • restore_callback_states_from_checkpoint: False
  • full_determinism: False
  • seed: 42
  • data_seed: None
  • use_cpu: False
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • parallelism_config: None
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • dataloader_prefetch_factor: None
  • remove_unused_columns: True
  • label_names: None
  • train_sampling_strategy: random
  • length_column_name: length
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • ddp_backend: None
  • ddp_timeout: 1800
  • fsdp: []
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • deepspeed: None
  • debug: []
  • skip_memory_metrics: True
  • do_predict: False
  • resume_from_checkpoint: None
  • warmup_ratio: None
  • local_rank: -1
  • prompts: None
  • batch_sampler: no_duplicates
  • multi_dataset_batch_sampler: round_robin
  • router_mapping: {}
  • learning_rate_mapping: {}

Training Logs

Epoch Step Training Loss val_cosine_accuracy
0.1484 500 2.9202 -
0.2967 1000 2.6123 -
0.4451 1500 2.5297 -
0.5935 2000 2.4455 -
0.7418 2500 2.3886 -
0.8902 3000 2.3452 -
1.0 3370 - 0.6140
2.0 6740 - 0.5615
3.0 10110 - 0.5676

The validation cosine_accuracy reported during training is computed on relaxed-threshold validation triplets (positive distance ≤ 3, negative distance ≥ 15), which is why it is lower than the strict-threshold test accuracy of 75.95% reported in the Evaluation section. The best checkpoint (epoch 1) is saved as the final model.

Framework Versions

  • Python: 3.12.10
  • Sentence Transformers: 5.3.0
  • Transformers: 5.3.0
  • PyTorch: 2.12.0.dev20260316+cu128
  • Accelerate: 1.13.0
  • Datasets: 4.7.0
  • Tokenizers: 0.22.2

Citation

BibTeX

This Model

@misc{kadiyala2026personality,
    title  = {Personality-Aware Embeddings: Fine-Tuning Transformers with Self-Reported OCEAN Scores for Author Similarity Retrieval},
    author = {Kadiyala, Likith and Sermet, Yusuf and Demir, Ibrahim},
    year   = {2026},
    note   = {Manuscript in preparation},
    url    = {https://huggingface.co/NextGen-Institute/big5-personality-mpnet}
}

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title     = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author    = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month     = "11",
    year      = "2019",
    publisher = "Association for Computational Linguistics",
    url       = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title         = {Efficient Natural Language Response Suggestion for Smart Reply},
    author        = {Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year          = {2017},
    eprint        = {1705.00652},
    archivePrefix = {arXiv},
    primaryClass  = {cs.CL}
}

Psychometric NLP Testbed

@inproceedings{abbasi2021constructing,
    title     = "Constructing a Psychometric Testbed for Fair Natural Language Processing",
    author    = "Abbasi, Ahmed and Dobolyi, David and Lalor, John P. and Netemeyer, Richard G. and Smith, Kendall and Yang, Yi",
    booktitle = "Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing",
    year      = "2021",
    pages     = "3748--3758"
}

License

This model is released under the Apache 2.0 License, matching the license of the base model sentence-transformers/all-mpnet-base-v2.

Downloads last month
5
Safetensors
Model size
0.1B params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for NextGen-Institute/big5-personality-mpnet

Finetuned
(416)
this model

Papers for NextGen-Institute/big5-personality-mpnet

Evaluation results