| --- |
| license: mit |
| --- |
| FireGenEmbedder |
|
|
| FireGenEmbedder is a fine-tuned version of the MiniLM model, specifically adapted for sequence classification tasks. The model has been fine-tuned on the Stanford Natural Language Inference (SNLI) dataset to predict the relationship between two sentences, classifying them into three categories: Entailment, Neutral, and Contradiction. It is designed for applications in legal and other domains requiring inference tasks. |
|
|
| Model Details |
|
|
| Base Model: sentence-transformers/all-MiniLM-L6-v2 |
|
|
| Fine-tuned Dataset: Stanford Natural Language Inference (SNLI) |
|
|
| Labels: |
|
|
| 0: Contradiction |
|
|
| 1: Neutral |
|
|
| 2: Entailment |
|
|
| Training Epochs: 3 |
|
|
| Batch Size: 16 (both train and eval) |
|
|
| Precision: Mixed precision for training on GPU |
|
|
| Model Usage |
|
|
| You can use this model to make inferences on sentence pairs by classifying their relationship. |
|
|
| Install Dependencies |
|
|
| To use this model, install the following libraries: |
|
|
| pip install transformers datasets sentence-transformers torch |
|
|
|
|
| Example Code |
|
|
| Here’s an example of how to load and use the FireGenEmbedder model for inference: |
|
|
| from transformers import AutoTokenizer, AutoModelForSequenceClassification |
| import torch |
|
|
| # Load the tokenizer and model |
| model_name = "path_to_firegenembedder_model" |
| tokenizer = AutoTokenizer.from_pretrained(model_name) |
| model = AutoModelForSequenceClassification.from_pretrained(model_name) |
|
|
| # Move model to device (GPU or CPU) |
| device = "cuda" if torch.cuda.is_available() else "cpu" |
| model.to(device) |
| |
| # Prepare input |
| premise = "The sky is blue." |
| hypothesis = "The sky is not blue." |
| |
| inputs = tokenizer(premise, hypothesis, return_tensors="pt", padding=True, truncation=True, max_length=128).to(device) |
| |
| # Inference |
| with torch.no_grad(): |
| outputs = model(**inputs) |
| predictions = torch.argmax(outputs.logits, dim=-1) |
| |
| # Print the prediction |
| labels = ["Contradiction", "Neutral", "Entailment"] |
| print(f"Prediction: {labels[predictions.item()]}") |
|
|
|
|
| Model Fine-Tuning Process |
|
|
| Data: The model was fine-tuned using the Stanford Natural Language Inference (SNLI) dataset. The SNLI dataset contains labeled pairs of sentences with three classes: Entailment, Neutral, and Contradiction. |
|
|
| Training: |
|
|
| The model was fine-tuned for 3 epochs with a batch size of 16 on a GPU. |
|
|
| The training used mixed precision for faster computation if a GPU was available. |
|
|
| The model is based on the MiniLM architecture, known for being lightweight and efficient, making it suitable for real-time inference tasks. |
|
|
| Post-Training: |
|
|
| The model was saved and zipped for easy distribution. |
|
|
| The tokenizer and model were saved to the directory: miniLM-legal-finetuned-SNLI. |
|
|
| Model Evaluation |
|
|
| The model was evaluated using the validation set from the SNLI dataset, and results can be accessed as follows: |
|
|
| # Load the model and evaluate |
| results = trainer.evaluate() |
| print(results) |
|
|
| Zipped Model |
|
|
| You can download the model as a zip file containing both the model weights and the tokenizer: |
|
|
| Download Model |
|
|
| Citation |
|
|
| If you use this model in your research or application, please cite the following: |
|
|
| @misc{firegenembedder, |
| author = {Your Name}, |
| title = {FireGenEmbedder: Fine-tuned MiniLM for Legal Inference Tasks}, |
| year = {2026}, |
| url = {Link to your Hugging Face model page}, |
| } |
