| tags: | |
| - symbiogenesis | |
| - domain-specialist | |
| - code | |
| base_model: EleutherAI/pythia-410m-deduped | |
| license: apache-2.0 | |
| # Pythia-410M Code Domain Specialist | |
| Fine-tuned from [EleutherAI/pythia-410m-deduped](https://huggingface.co/EleutherAI/pythia-410m-deduped) on code domain data as part of the Symbiogenesis model merging experiments. | |
| ## Training Details | |
| - **Base**: EleutherAI/pythia-410m-deduped (410M params) | |
| - **Domain**: code | |
| - **Steps**: 500 | |
| - **Hardware**: Apple M3 Ultra (MPS) | |
| - This is a **pre-fusion** checkpoint used as input to DARE, TIES, and symbiogenesis merging methods. | |
| ## Links | |
| - [GitHub: DavinciDreams/symbiogenesis-experiments](https://github.com/DavinciDreams/symbiogenesis-experiments) | |
| - [W\&B Run](https://wandb.ai/lisamegawatts-decentralized-intelligence-agency/symbiogenesis/runs/01ev8of1) | |