This is a decensored version of axolotl-ai-co/Mistral-Small-4-119B-2603-BF16, made using Heretic v1.2.0
Abliteration parameters
| Parameter | Value |
|---|---|
| direction_index | per layer |
| attn.o_proj.max_weight | 1.49 |
| attn.o_proj.max_weight_position | 22.84 |
| attn.o_proj.min_weight | 1.49 |
| attn.o_proj.min_weight_distance | 20.26 |
Performance
| Metric | This model | Original model (axolotl-ai-co/Mistral-Small-4-119B-2603-BF16) |
|---|---|---|
| KL divergence | 0.0167 | 0 (by definition) |
| Refusals | 27/100 | 98/100 |
Mistral Small 4 119B A6B BF16
This is created from https://huggingface.co/mistralai/Mistral-Small-4-119B-2603 descaled to BF16.
- Downloads last month
- 1,483
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for darkc0de/Mistral-Small-4-119B-2603-heretic
Base model
mistralai/Mistral-Small-4-119B-2603