---
license: apache-2.0
tags:
- knowledge-distillation
- pytorch
- transformers
base_model: unknown
---

# fokan/train-modle2

This model was created using knowledge distillation from the following teacher model(s):


## Model Description

A distilled model created using multi-modal knowledge distillation.

## Training Details

- **Teacher Models**: 
- **Distillation Strategy**: weighted
- **Training Steps**: 5000
- **Learning Rate**: 0.001

## Usage

```python
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("fokan/train-modle2")
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
```

## Created with

This model was created using the Multi-Modal Knowledge Distillation platform.