--- base_model: - mistralai/Mistral-Small-24B-Instruct-2501 - c4tdr0ut/grok-oss-Apollyon-24B - Casual-Autopsy/The-True-Abomination-24B - DavidAU/Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored - mradermacher/BlackXorDolphTronGOAT-GGUF - Nabbers1999/MS-24B-Bathory-GRPO - PocketDoc/Dans-DangerousWinds-V1.1.1-24b - ReadyArt/Broken-Tutu-24B-Transgression-v2.0 - TroyDoesAI/BlackSheep-24B library_name: transformers tags: - mergekit - merge - mistral - della - uncensored - profanity - roleplay - roleplaying - RP - Writing - creative - story - fiction - mistral - text - adventure - conversational widget: - text: "Paimon-24B-v2" output: url: https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/NJa8Ri4Abt5QRA44Y6U7a.png --- > [!CAUTION] > ⚠️ Warning: This model can produce narratives and RP that contain violent and graphic erotic content. Adjust your system prompt accordingly. Also, use **Mistral Tekken** template for best results. > # 👺 Paimon 24B v2 ![Paimon](https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/NJa8Ri4Abt5QRA44Y6U7a.png) This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). ## 🔀 Merge Details ### 🛠️ Merge Method This model was merged using the [DELLA](https://arxiv.org/abs/2406.11617) merge method using [mistralai/Mistral-Small-24B-Instruct-2501](https://huggingface.co/mistralai/Mistral-Small-24B-Instruct-2501) as a base. ## 🏭 Models Merged
v1: - mistralai/Mistral-Small-24B-Instruct-2501 - c4tdr0ut/grok-oss-Apollyon-24B - Casual-Autopsy/The-True-Abomination-24B - DavidAU/Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored - mradermacher/BlackXorDolphTronGOAT-GGUF v2: - mistralai/Mistral-Small-24B-Instruct-2501 - c4tdr0ut/grok-oss-Apollyon-24B - Casual-Autopsy/The-True-Abomination-24B - DavidAU/Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored - mradermacher/BlackXorDolphTronGOAT-GGUF - Nabbers1999/MS-24B-Bathory-GRPO - PocketDoc/Dans-DangerousWinds-V1.1.1-24b - ReadyArt/Broken-Tutu-24B-Transgression-v2.0 - TroyDoesAI/BlackSheep-24B v3: - mistralai/Mistral-Small-24B-Instruct-2501 - c4tdr0ut/grok-oss-Apollyon-24B - Casual-Autopsy/The-True-Abomination-24B - darkc0de/XortronCriminalComputingConfig - DavidAU/Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored - mradermacher/BlackXorDolphTronGOAT-GGUF - Nabbers1999/MS-24B-Bathory-GRPO - PocketDoc/Dans-DangerousWinds-V1.1.1-24b - ReadyArt/Broken-Tutu-24B-Transgression-v2.0 - TheDrummer/Cydonia-24B-v2 - TroyDoesAI/BlackSheep-24B Excluded from v3: - Undi95/MistralThinker-v1.1 - SlerpE/CardProjector-24B-v1 - spacewars123/Space-Wars-24B-v1.00a - arcee-ai/Arcee-Blitz - ArliAI/Mistral-Small-24B-ArliAI-RPMax-v1.4 - Mawdistical/Mawdistic-NightLife-24b
## ⚙️ Configuration
v1: ```yaml architecture: MistralForCausalLM models: - model: B:/24B/c4tdr0ut--grok-oss-Apollyon-24B parameters: weight: - filter: "lm_head" value: 1.0 - filter: "embed_tokens" value: 1.0 - value: 0.5 density: 0.9 epsilon: 0.09 - model: B:/24B/!models--mradermacher--BlackXorDolphTronGOAT-GGUF parameters: weight: - filter: "lm_head" value: 0.0 - filter: "embed_tokens" value: 0.0 - value: 0.5 density: 0.9 epsilon: 0.09 - model: B:/24B/DavidAU--Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored parameters: weight: - filter: "lm_head" value: 0.0 - filter: "embed_tokens" value: 0.0 - value: 0.5 density: 0.9 epsilon: 0.09 - model: B:/24B/Casual-Autopsy--The-True-Abomination-24B parameters: weight: - filter: "lm_head" value: 0.0 - filter: "embed_tokens" value: 0.0 - value: 0.5 density: 0.9 epsilon: 0.09 merge_method: della base_model: B:/24B/mistralai--Mistral-Small-24B-Instruct-2501 parameters: lambda: 1.0 normalize: false int8_mask: false rescale: true dtype: float32 out_dtype: bfloat16 tokenizer: source: union chat_template: auto ``` v2: ```yaml architecture: MistralForCausalLM models: - model: B:/24B/c4tdr0ut--grok-oss-Apollyon-24B parameters: weight: - filter: "lm_head" value: 0.0 - filter: "embed_tokens" value: 0.0 - value: 0.25 density: 0.9 epsilon: 0.09 - model: B:/24B/!models--mradermacher--BlackXorDolphTronGOAT-GGUF parameters: weight: - filter: "lm_head" value: 0.0 - filter: "embed_tokens" value: 0.0 - value: 0.25 density: 0.9 epsilon: 0.09 - model: B:/24B/!models--Nabbers1999--MS-24B-Bathory-GRPO parameters: weight: - filter: "lm_head" value: 0.0 - filter: "embed_tokens" value: 0.0 - value: 0.25 density: 0.9 epsilon: 0.09 - model: B:/24B/!models--ReadyArt--Broken-Tutu-24B-Transgression-v2.0 parameters: weight: - filter: "lm_head" value: 0.0 - filter: "embed_tokens" value: 0.0 - value: 0.25 density: 0.9 epsilon: 0.09 - model: B:/24B/!models--PocketDoc--Dans-DangerousWinds-V1.1.1-24b parameters: weight: - filter: "lm_head" value: 0.0 - filter: "embed_tokens" value: 0.0 - value: 0.25 density: 0.9 epsilon: 0.09 - model: B:/24B/TroyDoesAI--BlackSheep-24B parameters: weight: - filter: "lm_head" value: 1.0 - filter: "embed_tokens" value: 1.0 - value: 0.25 density: 0.9 epsilon: 0.09 - model: B:/24B/Casual-Autopsy--The-True-Abomination-24B parameters: weight: - filter: "lm_head" value: 0.0 - filter: "embed_tokens" value: 0.0 - value: 0.25 density: 0.9 epsilon: 0.09 - model: B:/24B/DavidAU--Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored parameters: weight: - filter: "lm_head" value: 0.0 - filter: "embed_tokens" value: 0.0 - value: 0.25 density: 0.9 epsilon: 0.09 merge_method: della base_model: B:/24B/!models--mistralai--Mistral-Small-24B-Instruct-2501 parameters: lambda: 1.0 normalize: false int8_mask: false rescale: true dtype: float32 out_dtype: bfloat16 tokenizer: source: union chat_template: auto ``` v3: ```yaml architecture: MistralForCausalLM models: - model: B:/24B/c4tdr0ut--grok-oss-Apollyon-24B parameters: weight: - filter: "lm_head" value: 0.0 - filter: "embed_tokens" value: 0.0 - value: 0.2 density: 0.9 epsilon: 0.09 - model: B:/24B/!models--mradermacher/BlackXorDolphTronGOAT-GGUF parameters: weight: - filter: "lm_head" value: 0.0 - filter: "embed_tokens" value: 0.0 - value: 0.2 density: 0.9 epsilon: 0.09 - model: B:/24B/!models--Nabbers1999--MS-24B-Bathory-GRPO parameters: weight: - filter: "lm_head" value: 0.0 - filter: "embed_tokens" value: 0.0 - value: 0.2 density: 0.9 epsilon: 0.09 - model: B:/24B/TheDrummer--Cydonia-24B-v2 parameters: weight: - filter: "lm_head" value: 0.0 - filter: "embed_tokens" value: 0.0 - value: 0.2 density: 0.9 epsilon: 0.09 - model: B:/24B/!models--ReadyArt--Broken-Tutu-24B-Transgression-v2.0 parameters: weight: - filter: "lm_head" value: 0.0 - filter: "embed_tokens" value: 0.0 - value: 0.2 density: 0.9 epsilon: 0.09 - model: B:/24B/!models--PocketDoc--Dans-DangerousWinds-V1.1.1-24b parameters: weight: - filter: "lm_head" value: 0.0 - filter: "embed_tokens" value: 0.0 - value: 0.2 density: 0.9 epsilon: 0.09 - model: B:/24B/TroyDoesAI--BlackSheep-24B parameters: weight: - filter: "lm_head" value: 1.0 - filter: "embed_tokens" value: 1.0 - value: 0.2 density: 0.9 epsilon: 0.09 - model: B:/24B/Casual-Autopsy--The-True-Abomination-24B parameters: weight: - filter: "lm_head" value: 0.0 - filter: "embed_tokens" value: 0.0 - value: 0.2 density: 0.9 epsilon: 0.09 - model: B:/24B/!models--darkc0de--XortronCriminalComputingConfig parameters: weight: - filter: "lm_head" value: 0.0 - filter: "embed_tokens" value: 0.0 - value: 0.2 density: 0.9 epsilon: 0.09 - model: B:/24B/DavidAU--Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored parameters: weight: - filter: "lm_head" value: 0.0 - filter: "embed_tokens" value: 0.0 - value: 0.2 density: 0.9 epsilon: 0.09 merge_method: della base_model: B:/24B/!models--mistralai--Mistral-Small-24B-Instruct-2501 parameters: lambda: 1.0 normalize: false int8_mask: false rescale: true dtype: float32 out_dtype: bfloat16 tokenizer: source: union chat_template: auto ```
## 📋 Audit Reports
``` [Paimon v1 DELLA Audit] Layer: model.layers.24.mlp.down_proj.weight | Lambda=1.00 [BASE] !models--mistralai--Mistral-Small-24B-Instruct-250 !models--mradermacher--BlackXorDolphTronGOAT-GGUF : ████████ 16.8% (W:0.50 D:0.90 N:0.44 E:0.09) Casual-Autopsy--The-True-Abomination-24B : █████████████ 27.4% (W:0.50 D:0.90 N:0.72 E:0.09) DavidAU--Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-: █████████ 19.5% (W:0.50 D:0.90 N:0.51 E:0.09) c4tdr0ut--grok-oss-Apollyon-24B : ██████████████████ 36.3% (W:0.50 D:0.90 N:0.95 E:0.09) [Paimon v2 DELLA Audit] Layer: model.layers.24.mlp.down_proj.weight | Lambda=1.00 [BASE] !models--mistralai--Mistral-Small-24B-Instruct-250 !models--Nabbers1999--MS-24B-Bathory-GRPO : ██████ 12.4% (W:0.25 D:0.90 N:0.58 E:0.09) !models--PocketDoc--Dans-DangerousWinds-V1.1.1-24b: █████████ 19.4% (W:0.25 D:0.90 N:0.91 E:0.09) !models--ReadyArt--Broken-Tutu-24B-Transgression-v: ██ 4.6% (W:0.25 D:0.90 N:0.21 E:0.09) !models--mradermacher--BlackXorDolphTronGOAT-GGUF : ████ 9.4% (W:0.25 D:0.90 N:0.44 E:0.09) Casual-Autopsy--The-True-Abomination-24B : ███████ 15.3% (W:0.25 D:0.90 N:0.72 E:0.09) DavidAU--Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-: █████ 10.9% (W:0.25 D:0.90 N:0.51 E:0.09) TroyDoesAI--BlackSheep-24B : ███ 7.6% (W:0.25 D:0.90 N:0.36 E:0.09) c4tdr0ut--grok-oss-Apollyon-24B : ██████████ 20.4% (W:0.25 D:0.90 N:0.95 E:0.09) [Paimon v3 DELLA Audit] Layer: model.layers.24.mlp.down_proj.weight | Lambda=1.00 [BASE] !models--mistralai--Mistral-Small-24B-Instruct-250 !models--darkc0de--XortronCriminalComputingConfig : ███ 6.4% (W:0.20 D:0.90 N:0.08 E:0.09) !models--Nabbers1999--MS-24B-Bathory-GRPO : █████ 10.9% (W:0.20 D:0.90 N:0.14 E:0.09) !models--PocketDoc--Dans-DangerousWinds-V1.1.1-24b: ████████ 17.6% (W:0.20 D:0.90 N:0.23 E:0.09) !models--ReadyArt--Broken-Tutu-24B-Transgression-v: ██ 4.1% (W:0.20 D:0.90 N:0.05 E:0.09) !models--mradermacher--BlackXorDolphTronGOAT-GGUF : ████ 8.1% (W:0.20 D:0.90 N:0.11 E:0.09) Casual-Autopsy--The-True-Abomination-24B : ███████ 14.6% (W:0.20 D:0.90 N:0.19 E:0.09) DavidAU--Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-: ████ 9.4% (W:0.20 D:0.90 N:0.12 E:0.09) TheDrummer--Cydonia-24B-v2 : ██ 4.5% (W:0.20 D:0.90 N:0.06 E:0.09) TroyDoesAI--BlackSheep-24B : ███ 6.4% (W:0.20 D:0.90 N:0.08 E:0.09) c4tdr0ut--grok-oss-Apollyon-24B : █████████ 18.1% (W:0.20 D:0.90 N:0.23 E:0.09) --- MAGNITUDE ANALYSIS & DATA POINTS --- ID | Status | Delta Norm | Orig Size | Model Name ---------------------------------------------------------------------------------------------------- #1 | HIGH MAG | 2.6618 | 33554432 | c4tdr0ut--grok-oss-Apollyon-24B #2 | OK | 1.1587 | 33554432 | mradermacher--BlackXorDolphTronGOAT-GGUF #3 | OK | 0.5315 | 33554432 | Nabbers1999--MS-24B-Bathory-GRPO #4 | OK | 0.9909 | 33554432 | Undi95--MistralThinker-v1.1 #5 | OK | 0.6966 | 33554432 | TheDrummer--Cydonia-24B-v2 #6 | OK | 0.5555 | 33554432 | ReadyArt--Broken-Tutu-24B-Transgression-v2.0 #7 | OK | 0.8735 | 33554432 | spacewars123--Space-Wars-24B-v1.00a #8 | OK | 1.3078 | 33554432 | arcee-ai--Arcee-Blitz #9 | OK | 1.1599 | 33554432 | PocketDoc--Dans-DangerousWinds-V1.1.1-24b #10 | OK | 0.4419 | 33554432 | TroyDoesAI--BlackSheep-24B #11 | OK | 1.2262 | 33554432 | Casual-Autopsy--The-True-Abomination-24B #12 | OK | 0.4419 | 33554432 | !darkc0de--XortronCriminalComputingConfig #13 | OK | 0.9952 | 33554432 | DavidAU--Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored #14 | OK | 1.1321 | 33554432 | ArliAI--Mistral-Small-24B-ArliAI-RPMax-v1.4 #15 | HIGH MAG | 2.8904 | 33554432 | SlerpE--CardProjector-24B-v1 #16 | OK | 0.2650 | 33554432 | Mawdistical--Mawdistic-NightLife-24b ``` ![paimon_v3_audit](https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/NAA-E4S9dEGjFwgUeiErN.png) Note that 6 models were excluded from v3 due to overly distorting the merge and re-activating the original base_model's positivity bias.