Instructions to use DarkArtsForge/Paimon-24B-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DarkArtsForge/Paimon-24B-v2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="DarkArtsForge/Paimon-24B-v2")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("DarkArtsForge/Paimon-24B-v2")
model = AutoModelForCausalLM.from_pretrained("DarkArtsForge/Paimon-24B-v2", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use DarkArtsForge/Paimon-24B-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DarkArtsForge/Paimon-24B-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DarkArtsForge/Paimon-24B-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/DarkArtsForge/Paimon-24B-v2

SGLang

How to use DarkArtsForge/Paimon-24B-v2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DarkArtsForge/Paimon-24B-v2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DarkArtsForge/Paimon-24B-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DarkArtsForge/Paimon-24B-v2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DarkArtsForge/Paimon-24B-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use DarkArtsForge/Paimon-24B-v2 with Docker Model Runner:
```
docker model run hf.co/DarkArtsForge/Paimon-24B-v2
```

Paimon-24B-v2 / README.md

Naphula

Create README.md

7886ed4 verified 9 days ago

preview code

Raw

History Blame Contribute Delete

15.4 kB

	---
	base_model:
	- mistralai/Mistral-Small-24B-Instruct-2501
	- c4tdr0ut/grok-oss-Apollyon-24B
	- Casual-Autopsy/The-True-Abomination-24B
	- DavidAU/Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored
	- mradermacher/BlackXorDolphTronGOAT-GGUF
	- Nabbers1999/MS-24B-Bathory-GRPO
	- PocketDoc/Dans-DangerousWinds-V1.1.1-24b
	- ReadyArt/Broken-Tutu-24B-Transgression-v2.0
	- TroyDoesAI/BlackSheep-24B
	library_name: transformers
	tags:
	- mergekit
	- merge
	- mistral
	- della
	- uncensored
	- profanity
	- roleplay
	- roleplaying
	- RP
	- Writing
	- creative
	- story
	- fiction
	- mistral
	- text
	- adventure
	- conversational
	widget:
	- text: "Paimon-24B-v2"
	output:
	url: https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/NJa8Ri4Abt5QRA44Y6U7a.png
	---

	> [!CAUTION]
	> <span style="color:red; font-weight:bold">⚠️ Warning:</span> This model can produce narratives and RP that contain violent and graphic erotic content. Adjust your system prompt accordingly. Also, use Mistral Tekken template for best results.
	>

	# 👺 Paimon 24B v2

	![Paimon](https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/NJa8Ri4Abt5QRA44Y6U7a.png)

	This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).

	## 🔀 Merge Details
	### 🛠️ Merge Method

	This model was merged using the [DELLA](https://arxiv.org/abs/2406.11617) merge method using [mistralai/Mistral-Small-24B-Instruct-2501](https://huggingface.co/mistralai/Mistral-Small-24B-Instruct-2501) as a base.

	## 🏭 Models Merged

	<details>

	v1:
	- mistralai/Mistral-Small-24B-Instruct-2501
	- c4tdr0ut/grok-oss-Apollyon-24B
	- Casual-Autopsy/The-True-Abomination-24B
	- DavidAU/Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored
	- mradermacher/BlackXorDolphTronGOAT-GGUF

	v2:
	- mistralai/Mistral-Small-24B-Instruct-2501
	- c4tdr0ut/grok-oss-Apollyon-24B
	- Casual-Autopsy/The-True-Abomination-24B
	- DavidAU/Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored
	- mradermacher/BlackXorDolphTronGOAT-GGUF
	- Nabbers1999/MS-24B-Bathory-GRPO
	- PocketDoc/Dans-DangerousWinds-V1.1.1-24b
	- ReadyArt/Broken-Tutu-24B-Transgression-v2.0
	- TroyDoesAI/BlackSheep-24B

	v3:
	- mistralai/Mistral-Small-24B-Instruct-2501
	- c4tdr0ut/grok-oss-Apollyon-24B
	- Casual-Autopsy/The-True-Abomination-24B
	- darkc0de/XortronCriminalComputingConfig
	- DavidAU/Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored
	- mradermacher/BlackXorDolphTronGOAT-GGUF
	- Nabbers1999/MS-24B-Bathory-GRPO
	- PocketDoc/Dans-DangerousWinds-V1.1.1-24b
	- ReadyArt/Broken-Tutu-24B-Transgression-v2.0
	- TheDrummer/Cydonia-24B-v2
	- TroyDoesAI/BlackSheep-24B

	Excluded from v3:
	- Undi95/MistralThinker-v1.1
	- SlerpE/CardProjector-24B-v1
	- spacewars123/Space-Wars-24B-v1.00a
	- arcee-ai/Arcee-Blitz
	- ArliAI/Mistral-Small-24B-ArliAI-RPMax-v1.4
	- Mawdistical/Mawdistic-NightLife-24b

	</details>

	## ⚙️ Configuration

	<details>

	v1:
	```yaml
	architecture: MistralForCausalLM
	models:
	- model: B:/24B/c4tdr0ut--grok-oss-Apollyon-24B
	parameters:
	weight:
	- filter: "lm_head"
	value: 1.0
	- filter: "embed_tokens"
	value: 1.0
	- value: 0.5
	density: 0.9
	epsilon: 0.09
	- model: B:/24B/!models--mradermacher--BlackXorDolphTronGOAT-GGUF
	parameters:
	weight:
	- filter: "lm_head"
	value: 0.0
	- filter: "embed_tokens"
	value: 0.0
	- value: 0.5
	density: 0.9
	epsilon: 0.09
	- model: B:/24B/DavidAU--Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored
	parameters:
	weight:
	- filter: "lm_head"
	value: 0.0
	- filter: "embed_tokens"
	value: 0.0
	- value: 0.5
	density: 0.9
	epsilon: 0.09
	- model: B:/24B/Casual-Autopsy--The-True-Abomination-24B
	parameters:
	weight:
	- filter: "lm_head"
	value: 0.0
	- filter: "embed_tokens"
	value: 0.0
	- value: 0.5
	density: 0.9
	epsilon: 0.09
	merge_method: della
	base_model: B:/24B/mistralai--Mistral-Small-24B-Instruct-2501
	parameters:
	lambda: 1.0
	normalize: false
	int8_mask: false
	rescale: true
	dtype: float32
	out_dtype: bfloat16
	tokenizer:
	source: union
	chat_template: auto
	```

	v2:
	```yaml
	architecture: MistralForCausalLM
	models:
	- model: B:/24B/c4tdr0ut--grok-oss-Apollyon-24B
	parameters:
	weight:
	- filter: "lm_head"
	value: 0.0
	- filter: "embed_tokens"
	value: 0.0
	- value: 0.25
	density: 0.9
	epsilon: 0.09
	- model: B:/24B/!models--mradermacher--BlackXorDolphTronGOAT-GGUF
	parameters:
	weight:
	- filter: "lm_head"
	value: 0.0
	- filter: "embed_tokens"
	value: 0.0
	- value: 0.25
	density: 0.9
	epsilon: 0.09
	- model: B:/24B/!models--Nabbers1999--MS-24B-Bathory-GRPO
	parameters:
	weight:
	- filter: "lm_head"
	value: 0.0
	- filter: "embed_tokens"
	value: 0.0
	- value: 0.25
	density: 0.9
	epsilon: 0.09
	- model: B:/24B/!models--ReadyArt--Broken-Tutu-24B-Transgression-v2.0
	parameters:
	weight:
	- filter: "lm_head"
	value: 0.0
	- filter: "embed_tokens"
	value: 0.0
	- value: 0.25
	density: 0.9
	epsilon: 0.09
	- model: B:/24B/!models--PocketDoc--Dans-DangerousWinds-V1.1.1-24b
	parameters:
	weight:
	- filter: "lm_head"
	value: 0.0
	- filter: "embed_tokens"
	value: 0.0
	- value: 0.25
	density: 0.9
	epsilon: 0.09
	- model: B:/24B/TroyDoesAI--BlackSheep-24B
	parameters:
	weight:
	- filter: "lm_head"
	value: 1.0
	- filter: "embed_tokens"
	value: 1.0
	- value: 0.25
	density: 0.9
	epsilon: 0.09
	- model: B:/24B/Casual-Autopsy--The-True-Abomination-24B
	parameters:
	weight:
	- filter: "lm_head"
	value: 0.0
	- filter: "embed_tokens"
	value: 0.0
	- value: 0.25
	density: 0.9
	epsilon: 0.09
	- model: B:/24B/DavidAU--Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored
	parameters:
	weight:
	- filter: "lm_head"
	value: 0.0
	- filter: "embed_tokens"
	value: 0.0
	- value: 0.25
	density: 0.9
	epsilon: 0.09
	merge_method: della
	base_model: B:/24B/!models--mistralai--Mistral-Small-24B-Instruct-2501
	parameters:
	lambda: 1.0
	normalize: false
	int8_mask: false
	rescale: true
	dtype: float32
	out_dtype: bfloat16
	tokenizer:
	source: union
	chat_template: auto
	```

	v3:
	```yaml
	architecture: MistralForCausalLM
	models:
	- model: B:/24B/c4tdr0ut--grok-oss-Apollyon-24B
	parameters:
	weight:
	- filter: "lm_head"
	value: 0.0
	- filter: "embed_tokens"
	value: 0.0
	- value: 0.2
	density: 0.9
	epsilon: 0.09
	- model: B:/24B/!models--mradermacher/BlackXorDolphTronGOAT-GGUF
	parameters:
	weight:
	- filter: "lm_head"
	value: 0.0
	- filter: "embed_tokens"
	value: 0.0
	- value: 0.2
	density: 0.9
	epsilon: 0.09
	- model: B:/24B/!models--Nabbers1999--MS-24B-Bathory-GRPO
	parameters:
	weight:
	- filter: "lm_head"
	value: 0.0
	- filter: "embed_tokens"
	value: 0.0
	- value: 0.2
	density: 0.9
	epsilon: 0.09
	- model: B:/24B/TheDrummer--Cydonia-24B-v2
	parameters:
	weight:
	- filter: "lm_head"
	value: 0.0
	- filter: "embed_tokens"
	value: 0.0
	- value: 0.2
	density: 0.9
	epsilon: 0.09
	- model: B:/24B/!models--ReadyArt--Broken-Tutu-24B-Transgression-v2.0
	parameters:
	weight:
	- filter: "lm_head"
	value: 0.0
	- filter: "embed_tokens"
	value: 0.0
	- value: 0.2
	density: 0.9
	epsilon: 0.09
	- model: B:/24B/!models--PocketDoc--Dans-DangerousWinds-V1.1.1-24b
	parameters:
	weight:
	- filter: "lm_head"
	value: 0.0
	- filter: "embed_tokens"
	value: 0.0
	- value: 0.2
	density: 0.9
	epsilon: 0.09
	- model: B:/24B/TroyDoesAI--BlackSheep-24B
	parameters:
	weight:
	- filter: "lm_head"
	value: 1.0
	- filter: "embed_tokens"
	value: 1.0
	- value: 0.2
	density: 0.9
	epsilon: 0.09
	- model: B:/24B/Casual-Autopsy--The-True-Abomination-24B
	parameters:
	weight:
	- filter: "lm_head"
	value: 0.0
	- filter: "embed_tokens"
	value: 0.0
	- value: 0.2
	density: 0.9
	epsilon: 0.09
	- model: B:/24B/!models--darkc0de--XortronCriminalComputingConfig
	parameters:
	weight:
	- filter: "lm_head"
	value: 0.0
	- filter: "embed_tokens"
	value: 0.0
	- value: 0.2
	density: 0.9
	epsilon: 0.09
	- model: B:/24B/DavidAU--Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored
	parameters:
	weight:
	- filter: "lm_head"
	value: 0.0
	- filter: "embed_tokens"
	value: 0.0
	- value: 0.2
	density: 0.9
	epsilon: 0.09
	merge_method: della
	base_model: B:/24B/!models--mistralai--Mistral-Small-24B-Instruct-2501
	parameters:
	lambda: 1.0
	normalize: false
	int8_mask: false
	rescale: true
	dtype: float32
	out_dtype: bfloat16
	tokenizer:
	source: union
	chat_template: auto
	```

	</details>

	## 📋 Audit Reports

	<details>

	```
	[Paimon v1 DELLA Audit] Layer: model.layers.24.mlp.down_proj.weight \| Lambda=1.00
	[BASE] !models--mistralai--Mistral-Small-24B-Instruct-250
	!models--mradermacher--BlackXorDolphTronGOAT-GGUF : ████████ 16.8% (W:0.50 D:0.90 N:0.44 E:0.09)
	Casual-Autopsy--The-True-Abomination-24B : █████████████ 27.4% (W:0.50 D:0.90 N:0.72 E:0.09)
	DavidAU--Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-: █████████ 19.5% (W:0.50 D:0.90 N:0.51 E:0.09)
	c4tdr0ut--grok-oss-Apollyon-24B : ██████████████████ 36.3% (W:0.50 D:0.90 N:0.95 E:0.09)

	[Paimon v2 DELLA Audit] Layer: model.layers.24.mlp.down_proj.weight \| Lambda=1.00
	[BASE] !models--mistralai--Mistral-Small-24B-Instruct-250
	!models--Nabbers1999--MS-24B-Bathory-GRPO : ██████ 12.4% (W:0.25 D:0.90 N:0.58 E:0.09)
	!models--PocketDoc--Dans-DangerousWinds-V1.1.1-24b: █████████ 19.4% (W:0.25 D:0.90 N:0.91 E:0.09)
	!models--ReadyArt--Broken-Tutu-24B-Transgression-v: ██ 4.6% (W:0.25 D:0.90 N:0.21 E:0.09)
	!models--mradermacher--BlackXorDolphTronGOAT-GGUF : ████ 9.4% (W:0.25 D:0.90 N:0.44 E:0.09)
	Casual-Autopsy--The-True-Abomination-24B : ███████ 15.3% (W:0.25 D:0.90 N:0.72 E:0.09)
	DavidAU--Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-: █████ 10.9% (W:0.25 D:0.90 N:0.51 E:0.09)
	TroyDoesAI--BlackSheep-24B : ███ 7.6% (W:0.25 D:0.90 N:0.36 E:0.09)
	c4tdr0ut--grok-oss-Apollyon-24B : ██████████ 20.4% (W:0.25 D:0.90 N:0.95 E:0.09)

	[Paimon v3 DELLA Audit] Layer: model.layers.24.mlp.down_proj.weight \| Lambda=1.00
	[BASE] !models--mistralai--Mistral-Small-24B-Instruct-250
	!models--darkc0de--XortronCriminalComputingConfig : ███ 6.4% (W:0.20 D:0.90 N:0.08 E:0.09)
	!models--Nabbers1999--MS-24B-Bathory-GRPO : █████ 10.9% (W:0.20 D:0.90 N:0.14 E:0.09)
	!models--PocketDoc--Dans-DangerousWinds-V1.1.1-24b: ████████ 17.6% (W:0.20 D:0.90 N:0.23 E:0.09)
	!models--ReadyArt--Broken-Tutu-24B-Transgression-v: ██ 4.1% (W:0.20 D:0.90 N:0.05 E:0.09)
	!models--mradermacher--BlackXorDolphTronGOAT-GGUF : ████ 8.1% (W:0.20 D:0.90 N:0.11 E:0.09)
	Casual-Autopsy--The-True-Abomination-24B : ███████ 14.6% (W:0.20 D:0.90 N:0.19 E:0.09)
	DavidAU--Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-: ████ 9.4% (W:0.20 D:0.90 N:0.12 E:0.09)
	TheDrummer--Cydonia-24B-v2 : ██ 4.5% (W:0.20 D:0.90 N:0.06 E:0.09)
	TroyDoesAI--BlackSheep-24B : ███ 6.4% (W:0.20 D:0.90 N:0.08 E:0.09)
	c4tdr0ut--grok-oss-Apollyon-24B : █████████ 18.1% (W:0.20 D:0.90 N:0.23 E:0.09)

	--- MAGNITUDE ANALYSIS & DATA POINTS ---
	ID \| Status \| Delta Norm \| Orig Size \| Model Name
	----------------------------------------------------------------------------------------------------
	#1 \| HIGH MAG \| 2.6618 \| 33554432 \| c4tdr0ut--grok-oss-Apollyon-24B
	#2 \| OK \| 1.1587 \| 33554432 \| mradermacher--BlackXorDolphTronGOAT-GGUF
	#3 \| OK \| 0.5315 \| 33554432 \| Nabbers1999--MS-24B-Bathory-GRPO
	#4 \| OK \| 0.9909 \| 33554432 \| Undi95--MistralThinker-v1.1
	#5 \| OK \| 0.6966 \| 33554432 \| TheDrummer--Cydonia-24B-v2
	#6 \| OK \| 0.5555 \| 33554432 \| ReadyArt--Broken-Tutu-24B-Transgression-v2.0
	#7 \| OK \| 0.8735 \| 33554432 \| spacewars123--Space-Wars-24B-v1.00a
	#8 \| OK \| 1.3078 \| 33554432 \| arcee-ai--Arcee-Blitz
	#9 \| OK \| 1.1599 \| 33554432 \| PocketDoc--Dans-DangerousWinds-V1.1.1-24b
	#10 \| OK \| 0.4419 \| 33554432 \| TroyDoesAI--BlackSheep-24B
	#11 \| OK \| 1.2262 \| 33554432 \| Casual-Autopsy--The-True-Abomination-24B
	#12 \| OK \| 0.4419 \| 33554432 \| !darkc0de--XortronCriminalComputingConfig
	#13 \| OK \| 0.9952 \| 33554432 \| DavidAU--Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored
	#14 \| OK \| 1.1321 \| 33554432 \| ArliAI--Mistral-Small-24B-ArliAI-RPMax-v1.4
	#15 \| HIGH MAG \| 2.8904 \| 33554432 \| SlerpE--CardProjector-24B-v1
	#16 \| OK \| 0.2650 \| 33554432 \| Mawdistical--Mawdistic-NightLife-24b
	```

	![paimon_v3_audit](https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/NAA-E4S9dEGjFwgUeiErN.png)

	Note that 6 models were excluded from v3 due to overly distorting the merge and re-activating the original base_model's positivity bias.

	</details>