model_293375959_coca_giant.py

Model Overview

A giant-scale implementation of the coca architecture, built for matching tasks.

Architecture

• Architecture: coca
• Scale: giant
• Attention: linear
• Fusion strategy: concat mlp
• Task head: matching
• Activation: approx gelu
• Normalization: instancenorm
• Initialization: xavier uniform

Training

• Optimizer: lion
• LR scheduler: step

Files

• model_293375959_coca_giant.py — main artifact of this repository

License

See the license field above.