AventIQ-AI/drama_base_sentence_similarity

Model Details

Model Description

This is a Sentence Transformer model fine-tuned from facebook/drama-base. It maps sentences and paragraphs to a 768-dimensional dense vector space and can be used for:

✅ Semantic Textual Similarity
✅ Semantic Search
✅ Paraphrase Mining
✅ Text Classification
✅ Clustering

Model Type: Sentence Transformer Base Model: facebook/drama-base Maximum Sequence Length: 512 tokens Output Dimensionality: 768 dimensions Similarity Function: Cosine Similarity

📚 Model Sources

Sentence Transformers Documentation

Repository: Sentence Transformers on GitHub
Hugging Face Model Card
🛠 Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

💡 Usage

Direct Usage (Sentence Transformers)
First, install the required libraries:

pip install -U sentence-transformers torch
Then, load the model and run inference:

from sentence_transformers import SentenceTransformer
import torch

# Load FP16 Quantized Model
model = SentenceTransformer("your_model_name").to("cuda" if torch.cuda.is_available() else "cpu")

# Encode Sentences
sentences = [
    "Artificial Intelligence is evolving rapidly.",
    "Machine Learning is a subset of AI.",
    "This is a random sentence."
]

embeddings = model.encode(sentences)
print(embeddings.shape)  # Output: (3, 768)

# Compute Similarity
def get_similarity(emb1, emb2):
    return torch.nn.functional.cosine_similarity(torch.tensor(emb1), torch.tensor(emb2), dim=0).item()

similarity_score = get_similarity(embeddings[0], embeddings[1])
print(f"Similarity Score: {similarity_score:.4f}")

📊 Training Details

Training Dataset

Dataset: STS-B (Semantic Textual Similarity Benchmark)
Size: 5,749 training samples
Columns: sentence_0, sentence_1, label

Sample Statistics

sentence_0	sentence_1	label
Biostatistics in Public Health	Statistics	1
Vital Signs: Understanding What the Body Is Telling Us	Data Science	0
Camino a la Excelencia en Gestión de Proyectos	Cybersecurity	0

{
    "distance_metric": "SiameseDistanceMetric.COSINE_DISTANCE",
    "margin": 0.5,
    "size_average": true
}

🔧 Training Hyperparameters

Hyperparameter	Value
per_device_train_batch_size	16
per_device_eval_batch_size	16
learning_rate	2e-5
epochs	1
optimizer	AdamW
⚙ Framework Versions
Library	Version
Python	3.12.7
Sentence Transformers	3.4.1
Transformers	4.49.0
PyTorch	2.5.1+cu124
Accelerate	1.3.0
Datasets	3.2.0
Tokenizers	0.21.0