Kansal commited on
Commit
301328a
·
verified ·
1 Parent(s): f0204ee

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +101 -0
README.md ADDED
@@ -0,0 +1,101 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ # Model Card for Paraphrase Detection Model
3
+
4
+ This model is fine-tuned for the **paraphrase detection** task on the GLUE MRPC dataset. It determines whether two given sentences are paraphrases (i.e., if they have the same meaning or not). This is a binary classification task with the following labels:
5
+
6
+ - **1**: Paraphrase
7
+ - **0**: Not a paraphrase
8
+
9
+ ## Model Overview
10
+
11
+ - **Developer**: Parit Kasnal
12
+ - **Model Type**: Sequence Classification (Binary)
13
+ - **Language(s)**: English
14
+ - **Pre-trained Model**: BERT (bert-base-uncased)
15
+
16
+ ## Intended Use
17
+
18
+ This model is designed to assess whether two sentences convey the same meaning. It can be applied in various scenarios, including:
19
+
20
+ - **Duplicate Question Detection**: Identifying similar questions in QA systems.
21
+ - **Plagiarism Detection**: Detecting if content is copied and rephrased.
22
+ - **Summarization Alignment**: Matching sentences from summaries to the original content.
23
+
24
+ ## Example Usage
25
+
26
+ ```python
27
+ from transformers import AutoModelForSequenceClassification, AutoTokenizer
28
+ import torch
29
+
30
+ # Load the fine-tuned model and tokenizer
31
+ model = AutoModelForSequenceClassification.from_pretrained("Parit1/dummy")
32
+ tokenizer = AutoTokenizer.from_pretrained("Parit1/dummy")
33
+
34
+ def make_prediction(text1, text2):
35
+ device = "cuda" if torch.cuda.is_available() else "cpu"
36
+ inputs = tokenizer(text1, text2, truncation=True, padding=True, return_tensors="pt")
37
+ inputs = {k: v.to(device) for k, v in inputs.items()}
38
+ model.to(device)
39
+ with torch.no_grad():
40
+ outputs = model(**inputs)
41
+ logits = outputs.logits
42
+ prediction = torch.argmax(logits, dim=-1).item()
43
+ return prediction
44
+
45
+ # Example usage
46
+ text1 = "The quick brown fox jumps over the lazy dog."
47
+ text2 = "A fast brown fox leaps over a lazy dog."
48
+ prediction = make_prediction(text1, text2)
49
+ print(f"Prediction: {prediction}")
50
+ ```
51
+
52
+ ## Training Details
53
+
54
+ ### Training Data
55
+ The model was fine-tuned on the **GLUE MRPC** dataset, which contains pairs of sentences labeled as either paraphrases or not.
56
+
57
+ ### Training Procedure
58
+ - **Number of Epochs**: 2
59
+ - **Metrics Used**:
60
+ - Accuracy
61
+ - Precision
62
+ - Recall
63
+ - F1 Score
64
+
65
+ #### Training Logs (Summary)
66
+ - **Epoch 1**:
67
+ - Training: Avg Loss = 0.5443, Accuracy = 73.45%, Precision = 72.28%, Recall = 73.45%, F1 Score = 70.83%
68
+ - Testing: Avg Loss = 0.3976, Accuracy = 82.60%, Precision = 82.26%, Recall = 82.60%, F1 Score = 81.93%
69
+
70
+ - **Epoch 2**:
71
+ - Training: Avg Loss = 0.2756, Accuracy = 89.34%, Precision = 89.25%, Recall = 89.34%, F1 Score = 89.27%
72
+ - Testing: Avg Loss = 0.3596, Accuracy = 84.80%, Precision = 84.94%, Recall = 84.80%, F1 Score = 84.87%
73
+
74
+ ## Evaluation
75
+
76
+ ### Performance Metrics
77
+ The model's performance was evaluated using the following metrics:
78
+
79
+ - **Accuracy**: Percentage of correct predictions.
80
+ - **Precision**: Proportion of positive identifications that were actually correct.
81
+ - **Recall**: Proportion of actual positives that were correctly identified.
82
+ - **F1 Score**: The harmonic mean of Precision and Recall.
83
+
84
+ ### Test Set Results
85
+ - **Epoch 1**:
86
+ - Avg Loss: 0.3976
87
+ - Accuracy: 82.60%
88
+ - Precision: 82.26%
89
+ - Recall: 82.60%
90
+ - F1 Score: 81.93%
91
+
92
+ - **Epoch 2**:
93
+ - Avg Loss: 0.3596
94
+ - Accuracy: 84.80%
95
+ - Precision: 84.94%
96
+ - Recall: 84.80%
97
+ - F1 Score: 84.87%
98
+
99
+ ---
100
+
101
+ This updated model card improves clarity, structure, and consistency, providing a more detailed explanation of each section while maintaining a professional tone.