metadata
license: cc-by-4.0
tags:
- batchnorm
- bilinear
- coca
- linear
- linear-warmup
- mish
- multitask
- novograd
- small
- trunc-normal
model_270412283_coca_small.py
Model Overview
A small-scale implementation of the coca architecture, built for multitask tasks.
Architecture
- Architecture: coca
- Scale: small
- Attention: linear
- Fusion strategy: bilinear
- Task head: multitask
- Activation: mish
- Normalization: batchnorm
- Initialization: trunc normal
Training
- Optimizer: novograd
- LR scheduler: linear warmup
Files
model_270412283_coca_small.py— main artifact of this repository
License
See the license field above.