Quant of TheDrummer/Behemoth-R1-123B-v2 at 6bpw h6 in exl2 for tabbyapi.
Runs great on 5 x 3090 or equivilant at 32k ctx and tensor parallel (see included tabbyapi model config overrides) using Largestral R1 text completion presets.
- Downloads last month
- 8
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support
Model tree for S3CUR/Behemoth-R1-123B-v2-6bpw-h6-exl2
Base model
mistralai/Mistral-Large-Instruct-2411
Finetuned
TheDrummer/Behemoth-R1-123B-v2