A small Mixtral-style Mixture-of-Experts causal language model for pretraining research on a single consumer GPU.