Hugging Face's logo

msugimura
/
gatekeeper_agent_responding

Sentence Similarity
sentence-transformers
TensorBoard
Safetensors
Transformers
English
bert
feature-extraction
agent-routing
conversation-matching
text-embeddings-inference
Model card Files
xet
Metrics Community
gatekeeper_agent_responding
File size: 4,871 Bytes 
e3de4d4
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
 
---
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers
- agent-routing
- conversation-matching
language: en
license: apache-2.0
datasets:
- custom
metrics:
- cosine_similarity
base_model: sentence-transformers/all-MiniLM-L12-v2
---

# Gatekeeper Agent Responding Model

This is a fine-tuned [sentence-transformers](https://www.SBERT.net) model based on **all-MiniLM-L12-v2** that has been specifically trained for **agent routing and conversation matching**. The model determines whether agents should respond to conversations based on semantic similarity.

## Model Details

### Base Model
- **Base Model**: [sentence-transformers/all-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L12-v2)
- **Model Architecture**: MiniLM-L12 (Microsoft)
- **Embedding Dimensions**: 384
- **Max Sequence Length**: 256 tokens

### Training Data
The model was fine-tuned on two custom datasets using triplet training:
- **semantic_triplet_training_data_round1.pkl**: 469 samples
- **inverse_semantic_triplet_training_data.pkl**: 475 samples

Each sample contains:
- `anchor`: Conversation text or agent description
- `positive`: Similar/relevant text to the anchor
- `negative`: Dissimilar/irrelevant text to the anchor

### Training Configuration
- **Loss Function**: MultipleNegativesRankingLoss
- **Batch Size**: 16
- **Learning Rate**: 2e-5
- **Epochs**: 1
- **Warmup Ratio**: 0.1
- **Training Framework**: sentence-transformers v2.7.0+

### Performance
Evaluation results on held-out test sets:
- **Semantic Triplets Accuracy**: 97.87%
- **Inverse Semantic Triplets Accuracy**: 100.00%

## Usage

### Direct Usage (Sentence Transformers)
```python
from sentence_transformers import SentenceTransformer

# Load the model
model = SentenceTransformer('msugimura/gatekeeper_agent_responding')

# Example: Agent routing for conversation
conversation = "I've been feeling anxious and need help with stress management"
agent_descriptions = [
    "Licensed therapist specializing in anxiety and stress management",
    "Fitness trainer who creates workout routines for stress relief",
    "Financial advisor who helps with investment planning"
]

# Get embeddings
conversation_embedding = model.encode(conversation)
agent_embeddings = model.encode(agent_descriptions)

# Calculate similarities
from sentence_transformers.util import cos_sim
similarities = cos_sim(conversation_embedding, agent_embeddings)

print("Similarity scores:", similarities)
# Expected: Highest similarity with the therapist
```

### API Usage (Portcullis Service)
```python
import requests

# Example API call to Portcullis service
response = requests.post("http://localhost:8000/should_agents_respond", json={
    "conversation": "I've been feeling anxious and need help",
    "agent_descriptions": [
        "Licensed therapist specializing in anxiety treatment",
        "Fitness trainer for workout routines",
        "Financial advisor for investments"
    ],
    "threshold": 0.4
})

result = response.json()
print("Qualified agents:", result["qualified_agents"])
```

## Intended Use Cases

1. **Agent Routing**: Automatically route conversations to appropriate specialist agents
2. **Conversation Matching**: Match user queries with relevant service providers
3. **Semantic Search**: Find similar conversations or agent descriptions
4. **Content Recommendation**: Recommend agents based on conversation context

## Limitations

- **Domain Specific**: Optimized for agent-conversation matching scenarios
- **English Only**: Trained primarily on English text
- **Context Length**: Limited to 256 tokens per input
- **Training Data**: Performance depends on similarity to training domain

## Technical Details

### Model Architecture
```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
  (2): Normalize()
)
```

### Training Process
1. **Data Preprocessing**: Cleaned triplet datasets, removed extraneous columns
2. **Multi-Dataset Training**: Combined training on both semantic and inverse semantic data
3. **Loss Function**: MultipleNegativesRankingLoss with in-batch negatives
4. **Evaluation**: TripletEvaluator on held-out validation sets

## Citation

If you use this model, please cite:

```bibtex
@misc{gatekeeper_agent_responding_2024,
  title={Gatekeeper Agent Responding Model},
  author={Michael Sugimura},
  year={2024},
  publisher={Hugging Face},
  url={https://huggingface.co/msugimura/gatekeeper_agent_responding}
}
```

## Contact

For questions or issues, please contact [your-email] or open an issue in the model repository.