AbstractPhil's picture
Licence: weights CC BY-NC-SA 4.0 (non-commercial); code, design and training process MIT
31e5d2d verified
Raw History Blame Contribute Delete
546 Bytes
This repository includes code adapted from amoe-lora (https://github.com/AbstractEyes/amoe-lora),
Copyright 2026 AbstractPhil, licensed under the Apache License, Version 2.0 (LICENSE-APACHE): the amoe-lora tree in each code snapshot.
The training code's Newton-Schulz orthogonalization (geolip/alephllm/train/optim.py) uses the quintic coefficients of
modded-nanogpt (https://github.com/KellerJordan/modded-nanogpt), Copyright (c) 2024 Keller Jordan, MIT License (its permission notice is the MIT text in LICENSE.md, under that copyright line).