Mocov3 experiments: Classification
Overview
This repository is a compact, custom PyTorch implementation of Mocov3 for Classification. The huge configuration is intended for code review, smoke tests, and small controlled experiments rather than as a production-ready pretrained release.
The whole setup fits in a single module on purpose.
Architecture
| Item | Value |
|---|---|
| Architecture | Mocov3 |
| Scale | huge |
| Attention | sparse |
| Fusion | concat mlp |
| Activation | relu |
| Normalization | rmsnorm |
Repository status
One Python module contains the model and a runnable example.
Generated configuration lives in config.json; the recipe in training_args.json.
The checkpoint is an initialization artifact and should not be read as a trained model.
Scores are deliberately absent.
Contact
Questions or corrections: open a discussion in this repository.
Limitations
The initialization checkpoint has not been trained or audited for robustness, fairness, or domain transfer. The implementation should be treated as an experimental starting point. Results from a future trained checkpoint must be documented separately from the defaults shipped here.
Roadmap
Next steps, in no particular order: a cleaner entry point, input-shape docs, and a small evaluation script if time permits.
Default experiment recipe
Defaults: adafactor optimizer, exponential schedule. Treat these as starting points; a real run should sweep learning rate and batch size before drawing conclusions.
Files
| File | Purpose |
|---|---|
train.py |
primary artifact |
README.md |
this documentation |
config.json |
architecture configuration |
training_args.json |
default experiment settings |
model.safetensors |
initialization checkpoint |
License
Distributed under the cc-by-4.0 license. External datasets keep their own terms — check them separately.
- Downloads last month
- 18