|
|
--- |
|
|
library_name: transformers |
|
|
tags: |
|
|
- nli |
|
|
- natural-language-inference |
|
|
- knowledge-distillation |
|
|
- srl |
|
|
- self-reflective-learning |
|
|
- biomedical-nlp |
|
|
- aethermind |
|
|
- student-model |
|
|
license: mit |
|
|
datasets: |
|
|
- your-dataset-name-here |
|
|
language: |
|
|
- en |
|
|
pipeline_tag: text-classification |
|
|
model_name: AetherMind_SRL |
|
|
model_creator: samerzaher80 |
|
|
--- |
|
|
|
|
|
# 📘 AetherMind_SRL – Self-Reflective Learning NLI Model |
|
|
Author: Samer S. Najm (Sam) |
|
|
Organization: AetherMind Project |
|
|
Model Type: Knowledge-Distilled Transformer (Student Model) |
|
|
Domain: Natural Language Inference (NLI) + Medical Reasoning (ADNI SRL) |
|
|
|
|
|
## 🚀 Overview |
|
|
AetherMind_SRL is the 12th-round refined version of AetherMind’s knowledge-distilled student model, trained using self-reflective learning (SRL), knowledge distillation, ADNI medical contradictions, and general-domain NLI datasets. |
|
|
|
|
|
## 💡 Highlights |
|
|
- Improved contradiction detection |
|
|
- Strong general NLI performance |
|
|
- Lightweight and efficient |
|
|
- SRL-based iterative refinement |
|
|
|
|
|
## 📊 Evaluation (Round 12 Final) |
|
|
| Dataset | Accuracy | Macro F1 | Samples | |
|
|
|-----------|----------|----------|---------| |
|
|
| SNLI | 89.64% | 89.55% | 9,824 | |
|
|
| MNLI-M | 90.20% | 90.00% | 9,815 | |
|
|
| MNLI-MM | 89.61% | 89.35% | 9,832 | |
|
|
| ANLI R1 | 79.90% | 79.89% | 1,000 | |
|
|
| ANLI R2 | 67.50% | 67.35% | 1,000 | |
|
|
| ANLI R3 | 67.33% | 66.81% | 1,200 | |
|
|
|
|
|
## 🧠 Self-Reflective Learning (SRL) |
|
|
1. Train base model |
|
|
2. Extract errors |
|
|
3. Correct and retrain |
|
|
4. Stabilize via KD + SRL loops |
|
|
|
|
|
## 🔥 Teacher → Student Distillation |
|
|
Teacher: microsoft/deberta-v3-base |
|
|
Student: AetherMind_SRL |
|
|
|
|
|
## 🛠 Usage Example |
|
|
```python |
|
|
from transformers import AutoTokenizer, AutoModelForSequenceClassification |
|
|
import torch |
|
|
|
|
|
model_id = "samerzaher80/AetherMind_SRL" |
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id) |
|
|
model = AutoModelForSequenceClassification.from_pretrained(model_id).cuda() |
|
|
|
|
|
premise = "The patient scored 28 on the MMSE last year." |
|
|
hypothesis = "The patient shows signs of cognitive decline." |
|
|
|
|
|
inputs = tokenizer(premise, hypothesis, return_tensors="pt").to("cuda") |
|
|
with torch.no_grad(): |
|
|
logits = model(**inputs).logits |
|
|
predicted = torch.argmax(logits, dim=-1).item() |
|
|
|
|
|
labels = ["entailment", "neutral", "contradiction"] |
|
|
print("Prediction:", labels[predicted]) |
|
|
``` |
|
|
|
|
|
## 🔧 Included Python Files |
|
|
- evaluate_round12.py |
|
|
- inference_srl_round12.py |
|
|
- train_round12_srl_kd.py |
|
|
- build_anli_global_error_buffer_round1.py |
|
|
- analyze_anli_errors_round1.py |
|
|
- srl_finetune_round5_smart.py |
|
|
|
|
|
## 🧩 Metadata |
|
|
```json |
|
|
{ |
|
|
"tags": [ |
|
|
"natural-language-inference", |
|
|
"knowledge-distillation", |
|
|
"biomedical-nlp", |
|
|
"aethermind", |
|
|
"nli", |
|
|
"self-reflective-learning", |
|
|
"transformers" |
|
|
] |
|
|
} |
|
|
``` |
|
|
|
