---
license: mit
tags:
- approx-gelu
- classification
- dino
- layernorm
- low-rank
- rmsprop
- sparse
- step
- trunc-normal
- xlarge
---

# pipeline.py

## Model Overview

A **xlarge**-scale implementation of the **dino** architecture, built for **classification** tasks.

## Architecture

- **Architecture**: dino
- **Scale**: xlarge
- **Attention**: sparse
- **Fusion strategy**: low rank
- **Task head**: classification
- **Activation**: approx gelu
- **Normalization**: layernorm
- **Initialization**: trunc normal

## Training

- **Optimizer**: rmsprop
- **LR scheduler**: step

## Files

- `pipeline.py` — main artifact of this repository

## License

See the license field above.