Add README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,64 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
tags:
|
| 4 |
+
- pytorch
|
| 5 |
+
- efficientformer
|
| 6 |
+
- classification
|
| 7 |
+
---
|
| 8 |
+
|
| 9 |
+
# Efficientformer for Classification
|
| 10 |
+
|
| 11 |
+
## Overview
|
| 12 |
+
|
| 13 |
+
Working implementation of **Efficientformer** for **Classification** using a **base** configuration. The repository focuses on transparent code and repeatable smoke tests; benchmark claims are deliberately omitted.
|
| 14 |
+
|
| 15 |
+
## Repository status
|
| 16 |
+
|
| 17 |
+
- The Python file contains the model and runnable example or training entry point.
|
| 18 |
+
- `config.json` records the generated architecture settings.
|
| 19 |
+
- `training_args.json` records the default experiment recipe.
|
| 20 |
+
- `model.safetensors` is a valid initialization checkpoint for smoke tests; it is **not** presented as a trained benchmark checkpoint.
|
| 21 |
+
- No benchmark score is claimed in this repository.
|
| 22 |
+
|
| 23 |
+
## Architecture
|
| 24 |
+
|
| 25 |
+
| Item | Value |
|
| 26 |
+
|---|---|
|
| 27 |
+
| Architecture | Efficientformer |
|
| 28 |
+
| Scale | base |
|
| 29 |
+
| Attention | linear |
|
| 30 |
+
| Fusion | tucker |
|
| 31 |
+
| Activation | relu |
|
| 32 |
+
| Normalization | scalenorm |
|
| 33 |
+
|
| 34 |
+
## Default experiment recipe
|
| 35 |
+
|
| 36 |
+
The included configuration uses **lamb** with a **cosine** schedule. These are starting values in the script, not evidence of a completed run. For a meaningful evaluation, train all baselines with the same data exposure, tuning budget, and random seeds.
|
| 37 |
+
|
| 38 |
+
## Quick check
|
| 39 |
+
|
| 40 |
+
```bash
|
| 41 |
+
python run.py --help
|
| 42 |
+
```
|
| 43 |
+
|
| 44 |
+
Inspect the script's `__main__` block for its generated smoke-test example. Because this is a custom implementation, generic automatic loading APIs require an explicit adapter before use.
|
| 45 |
+
|
| 46 |
+
## Evaluation guidance
|
| 47 |
+
|
| 48 |
+
A useful first evaluation would use **a task-specific labeled split**, report the task metric across at least three seeds, and include a matched-capacity baseline. Keep training logs and environment versions with any published result.
|
| 49 |
+
|
| 50 |
+
## Limitations
|
| 51 |
+
|
| 52 |
+
The initialization checkpoint has not been trained or audited for robustness, fairness, or domain transfer. The implementation should be treated as an experimental starting point. Results from a future trained checkpoint must be documented separately from the defaults shipped here.
|
| 53 |
+
|
| 54 |
+
## Files
|
| 55 |
+
|
| 56 |
+
- `run.py` — primary artifact
|
| 57 |
+
- `README.md` — this documentation
|
| 58 |
+
- `config.json` — architecture configuration
|
| 59 |
+
- `training_args.json` — default experiment settings
|
| 60 |
+
- `model.safetensors` — initialization checkpoint
|
| 61 |
+
|
| 62 |
+
## License
|
| 63 |
+
|
| 64 |
+
Released under **mit**. Review the source-data terms separately when this repository is used with external datasets.
|