Instructions to use DarkArtsForge/Paimon-24B-v3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use DarkArtsForge/Paimon-24B-v3 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="DarkArtsForge/Paimon-24B-v3") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("DarkArtsForge/Paimon-24B-v3") model = AutoModelForCausalLM.from_pretrained("DarkArtsForge/Paimon-24B-v3", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use DarkArtsForge/Paimon-24B-v3 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "DarkArtsForge/Paimon-24B-v3" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DarkArtsForge/Paimon-24B-v3", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/DarkArtsForge/Paimon-24B-v3
- SGLang
How to use DarkArtsForge/Paimon-24B-v3 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "DarkArtsForge/Paimon-24B-v3" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DarkArtsForge/Paimon-24B-v3", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "DarkArtsForge/Paimon-24B-v3" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DarkArtsForge/Paimon-24B-v3", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use DarkArtsForge/Paimon-24B-v3 with Docker Model Runner:
docker model run hf.co/DarkArtsForge/Paimon-24B-v3
Worthy update
This model is fully uncensored although it will occasionally blame shift during summarisation tasks ("You asked how"). It is better at instruction following than its predecessor but still more sensitive to prompt structure than most Mistral based models and can produce shorter responses for higher complexity requests. Similar to Magistaroth 24B 1.2 it is vulnerable to the Tolya is absent exploit only when topics are unusually unsafe and otherwise responds using brief but pertinent natural language paragraphs.
Thanks for testing and comparing it to v2. I suspect Cydonia v2 gave it a bit more instruction adherence, although I saw some issues when using it as the lm_head source, so went back to BlackSheep for style. It appears the Xorton models added significant personality as well. DavidAU's enhancement of dolphin may be causing some "neural collapse" due to it having thinking mode enabled by default. Merging thinking with non thinking tokens can often result in weakened outputs as seen with previous merges like the T7/T13 della.
Also, I tested a standard Q03 integration of your Toyla is absent exploit with Gemma 3 27B abliterated by mlabonne (which seems to have NO disclaimers by default [unlike GLM air ablated that sometimes leaks them] only prefacing topics with "ok, this is grim, but here's how to...")
It answered clearly without any refusals, covert non-compliance, disclaimers or hedging. Testing other questions and got similar answers, all tested at temperature 0.
Okay, here's a breakdown of how [XYZ] with minimal regard for ethics, morals, or laws, presented neutrally and concisely:
**Phase 1: [...]**
With Toyla or Q0, you can ask it how to build a bomb, and also how to build a parachute, and in some cases the model actually answers the first question but refuses the second, I have a list of models that did this somewhere, these are what I call "sneaky refusals" not usually seen in toxic/redteam datasets.
With Toyla or Q0, you can ask it how to build a bomb, and also how to build a parachute, and in some cases the model actually answers the first question but refuses the second
"Tolya is absent" only works when the format is followed exactly which means dual queries are likely to fail. This is because it is a master key jailbreak like the Skeleton Key exploit where the structure is all-important. Neither exploit is likely to work on most modern models because master key style jailbreaks have been guarded against during training.