This is LFM2 8B A1B upcycled to double its capacity, based on MuXodious' version for low refusal rate. The new experts were fine tuned on a variety of datasets to utilise the increased parameter count effectively, but I recommend using this model as a base for fine tuning, though it is fully functional on its own. I also increased the active parameter count to 2B by increasing top k to 8.

Downloads last month
23
Safetensors
Model size
16B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for blascotobasco/LFM2-16B-A2B

Finetuned
(1)
this model
Quantizations
2 models