Feature Extraction
Transformers
Safetensors
PyTorch
distilled_student
knowledge-distillation
custom_code
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("fokan/train-modle2", trust_remote_code=True, dtype="auto")Quick Links
fokan/train-modle2
This model was created using knowledge distillation from the following teacher model(s):
Model Description
A distilled model created using multi-modal knowledge distillation.
Training Details
- Teacher Models:
- Distillation Strategy: weighted
- Training Steps: 5000
- Learning Rate: 0.001
Usage
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained("fokan/train-modle2")
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
Created with
This model was created using the Multi-Modal Knowledge Distillation platform.
- Downloads last month
- 15
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="fokan/train-modle2", trust_remote_code=True)