|
Download README.md from KatieHg74/efficientformer-multitask-final: direct link, hf CLI and curl.
- Browser
- Download file 2.41 kB
-
https://huggingface.co/KatieHg74/efficientformer-multitask-final/resolve/main/README.md
- Command line
-
hf download hf://KatieHg74/efficientformer-multitask-final/README.md
-
curl -L -o README.md https://huggingface.co/KatieHg74/efficientformer-multitask-final/resolve/main/README.md
2.41 kB
| license: apache-2.0 | |
| tags: | |
| - pytorch | |
| - efficientformer | |
| - multitask | |
| - wip | |
| - initialization | |
| # Efficientformer for Multitask | |
| ## Overview | |
| This is an experimental **Efficientformer** codebase for **Multitask**. It keeps the **tiny** setup intentionally manageable so architecture changes can be inspected before a full training run. | |
| --- | |
| ## Architecture | |
| - **Architecture**: Efficientformer | |
| - **Scale**: tiny | |
| - **Attention**: standard | |
| - **Fusion**: cross attention | |
| - **Activation**: relu | |
| - **Normalization**: batchnorm | |
| --- | |
| ## Training details | |
| - Weights initialized with a orthogonal scheme. | |
| - Runs on a single consumer GPU. | |
| - Full smoke test takes minutes, not hours. | |
| - No distributed setup required. | |
| --- | |
| ## Limitations | |
| The initialization checkpoint has not been trained or audited for robustness, fairness, or domain transfer. The implementation should be treated as an experimental starting point. Results from a future trained checkpoint must be documented separately from the defaults shipped here. | |
| --- | |
| ## Default experiment recipe | |
| The included configuration uses **lion** with a **linear warmup** schedule. These are starting values in the script, not evidence of a completed run. For a meaningful evaluation, train all baselines with the same data exposure, tuning budget, and random seeds. | |
| --- | |
| ## Quick check | |
| Run `python model.py --help` for the built-in smoke test. Because this is a custom implementation, generic automatic loading APIs require an explicit adapter before use. | |
| --- | |
| ## Roadmap | |
| Next steps, in no particular order: a cleaner entry point, input-shape docs, and a small evaluation script if time permits. | |
| --- | |
| ## Repository status | |
| One Python module contains the model and a runnable example. | |
| Generated configuration lives in `config.json`; the recipe in `training_args.json`. | |
| The checkpoint is an initialization artifact and should not be read as a trained model. | |
| Scores are deliberately absent. | |
| --- | |
| ## Files | |
| - `model.py` — primary artifact | |
| - `README.md` — this documentation | |
| - `config.json` — architecture configuration | |
| - `training_args.json` — default experiment settings | |
| - `model.safetensors` — initialization checkpoint | |
| --- | |
| ## License | |
| Released under **apache-2.0**. Review the source-data terms separately when this repository is used with external datasets. | |