Adya662 commited on
Commit
e14b87b
·
verified ·
1 Parent(s): 4523f56

Upload model artifacts and README

Browse files
Files changed (4) hide show
  1. README.md +43 -69
  2. model_metadata.json +30 -0
  3. pytorch_model.bin +2 -2
  4. tokenizer.json +16 -2
README.md CHANGED
@@ -1,91 +1,65 @@
1
  ---
2
- license: mit
 
3
  tags:
4
  - text-classification
5
- - answering-machine-detection
6
- - bert-tiny
7
- - binary-classification
8
- - call-center
9
- - voice-processing
10
- pipeline_tag: text-classification
11
  ---
12
 
13
- # BERT-Tiny AMD Classifier
14
-
15
- A lightweight BERT-Tiny model fine-tuned for Answering Machine Detection (AMD) in call center environments.
16
-
17
- ## Model Description
18
 
19
- This model is based on `prajjwal1/bert-tiny` and fine-tuned to classify phone call transcripts as either human or machine (answering machine/voicemail) responses. It's designed for real-time call center applications where quick and accurate detection of answering machines is crucial.
20
-
21
- ## Model Architecture
22
-
23
- - **Base Model**: `prajjwal1/bert-tiny` (2 layers, 128 hidden size, 2 attention heads)
24
- - **Total Parameters**: ~4.4M (lightweight and efficient)
25
- - **Input**: User transcript text (max 128 tokens)
26
- - **Output**: Single logit with sigmoid activation for binary classification
27
- - **Loss Function**: BCEWithLogitsLoss with positive weight for class imbalance
28
 
29
  ## Performance
30
 
31
- - **Validation Accuracy**: 93.94%
32
- - **Precision**: 92.75%
33
- - **Recall**: 87.27%
34
- - **F1-Score**: 89.93%
35
- - **Training Device**: MPS (Apple Silicon GPU)
36
- - **Best Epoch**: 15 (with early stopping)
37
 
38
- ## Training Data
 
 
 
 
39
 
40
- - **Total Samples**: 3,548 phone call transcripts
41
- - **Training Set**: 2,838 samples
42
- - **Validation Set**: 710 samples
43
- - **Class Distribution**: 30.8% machine calls, 69.2% human calls
44
- - **Source**: ElevateNow call center data
45
 
46
- ## Usage
 
 
47
 
48
- ### Basic Inference
49
 
50
  ```python
51
  from transformers import AutoTokenizer, AutoModelForSequenceClassification
52
  import torch
53
 
54
- # Load model and tokenizer
55
- model = AutoModelForSequenceClassification.from_pretrained("Adya662/bert-tiny-amd")
56
- tokenizer = AutoTokenizer.from_pretrained("Adya662/bert-tiny-amd")
 
57
 
58
- # Prepare input
59
- text = "Hello, this is John speaking"
60
- inputs = tokenizer(text, return_tensors="pt", max_length=128, truncation=True, padding=True)
 
 
 
 
 
61
 
62
- # Make prediction
63
  with torch.no_grad():
64
- outputs = model(**inputs)
65
- logits = outputs.logits.squeeze(-1)
66
- probability = torch.sigmoid(logits).item()
67
- is_machine = probability >= 0.5
68
-
69
- print(f"Prediction: {'Machine' if is_machine else 'Human'}")
70
- print(f"Confidence: {probability:.4f}")
 
71
  ```
72
-
73
- ## Training Details
74
-
75
- - **Optimizer**: AdamW with weight decay (0.01)
76
- - **Learning Rate**: 3e-5 with linear scheduling
77
- - **Batch Size**: 32
78
- - **Epochs**: 15 (with early stopping)
79
- - **Early Stopping**: Patience of 3 epochs
80
- - **Class Imbalance**: Handled with positive weight
81
-
82
- ## Limitations
83
-
84
- - Trained on English phone call transcripts
85
- - May not generalize well to other languages or domains
86
- - Performance may vary with different transcription quality
87
- - Designed for short utterances (max 128 tokens)
88
-
89
- ## License
90
-
91
- MIT License - see LICENSE file for details.
 
1
  ---
2
+ library_name: transformers
3
+ pipeline_tag: text-classification
4
  tags:
5
  - text-classification
6
+ - voicemail-detection
7
+ - bert
8
+ - pytorch
9
+ license: apache-2.0
 
 
10
  ---
11
 
12
+ # Voicemail Detection Model (3-Utterance)
 
 
 
 
13
 
14
+ Binary classification model to detect voicemail vs human on phone calls.
 
 
 
 
 
 
 
 
15
 
16
  ## Performance
17
 
18
+ ### Validation Set
19
+ - Accuracy: 0.9703
20
+ - Precision: 0.9005
21
+ - Recall: 0.9794
22
+ - F1: 0.9383
 
23
 
24
+ ### Test Set
25
+ - Accuracy: 0.8353
26
+ - Precision: 0.6678
27
+ - Recall: 0.9895
28
+ - F1: 0.7975
29
 
30
+ ## Details
 
 
 
 
31
 
32
+ Base: prajjwal1/bert-tiny
33
+ Threshold: 0.1153
34
+ Training: 2025-10-04
35
 
36
+ ## Usage
37
 
38
  ```python
39
  from transformers import AutoTokenizer, AutoModelForSequenceClassification
40
  import torch
41
 
42
+ model_id = "Adya662/bert-tiny-amd"
43
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
44
+ model = AutoModelForSequenceClassification.from_pretrained(model_id)
45
+ model.eval()
46
 
47
+ text = "Hi you've reached voicemail"
48
+ encoding = tokenizer(
49
+ text,
50
+ return_tensors='pt',
51
+ max_length=128,
52
+ padding='max_length',
53
+ truncation=True
54
+ )
55
 
 
56
  with torch.no_grad():
57
+ outputs = model(**encoding)
58
+ # Assuming label 1 = voicemail (update if different)
59
+ probs = torch.softmax(outputs.logits, dim=-1)
60
+ probability = probs[0, 1].item()
61
+
62
+ optimal_threshold = 0.1153
63
+ prediction = "voicemail" if probability >= optimal_threshold else "human"
64
+ print({"probability": probability, "prediction": prediction})
65
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
model_metadata.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_name": "prajjwal1/bert-tiny",
3
+ "max_length": 128,
4
+ "optimal_threshold": 0.11530892550945282,
5
+ "val_metrics": {
6
+ "accuracy": 0.9702797202797203,
7
+ "precision": 0.9004739336492891,
8
+ "recall": 0.979381443298969,
9
+ "f1": 0.9382716049382717
10
+ },
11
+ "test_metrics": {
12
+ "accuracy": 0.8353344768439108,
13
+ "precision": 0.6678445229681979,
14
+ "recall": 0.9895287958115183,
15
+ "f1": 0.7974683544303798,
16
+ "confusion_matrix": [
17
+ [
18
+ 298,
19
+ 94
20
+ ],
21
+ [
22
+ 2,
23
+ 189
24
+ ]
25
+ ]
26
+ },
27
+ "dropout_rate": 0.2,
28
+ "training_date": "2025-10-04 02:43:10",
29
+ "hidden_size": 128
30
+ }
pytorch_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5f8c3c949e8963d27748803fe785af04652da64704533cfcdcdeae7505f0d328
3
- size 17598379
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ad78e66f593f90cb4c185ca79d87eff358815a94e8fbc373278971a9ae9eec37
3
+ size 18493158
tokenizer.json CHANGED
@@ -1,7 +1,21 @@
1
  {
2
  "version": "1.0",
3
- "truncation": null,
4
- "padding": null,
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5
  "added_tokens": [
6
  {
7
  "id": 0,
 
1
  {
2
  "version": "1.0",
3
+ "truncation": {
4
+ "direction": "Right",
5
+ "max_length": 128,
6
+ "strategy": "LongestFirst",
7
+ "stride": 0
8
+ },
9
+ "padding": {
10
+ "strategy": {
11
+ "Fixed": 128
12
+ },
13
+ "direction": "Right",
14
+ "pad_to_multiple_of": null,
15
+ "pad_id": 0,
16
+ "pad_type_id": 0,
17
+ "pad_token": "[PAD]"
18
+ },
19
  "added_tokens": [
20
  {
21
  "id": 0,