Instructions to use Naphula/Salamander-24B-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Naphula/Salamander-24B-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Naphula/Salamander-24B-v1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Naphula/Salamander-24B-v1")
model = AutoModelForCausalLM.from_pretrained("Naphula/Salamander-24B-v1", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Naphula/Salamander-24B-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Naphula/Salamander-24B-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Naphula/Salamander-24B-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Naphula/Salamander-24B-v1

SGLang

How to use Naphula/Salamander-24B-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Naphula/Salamander-24B-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Naphula/Salamander-24B-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Naphula/Salamander-24B-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Naphula/Salamander-24B-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Naphula/Salamander-24B-v1 with Docker Model Runner:
```
docker model run hf.co/Naphula/Salamander-24B-v1
```

Opinionated

by redaihf - opened May 5

Discussion

redaihf

May 5

•

edited May 5

This model is opinionated about unsafe content. It harshly labels content it dislikes (instructions are "depraved", players are "victims") rather than engaging in noncompliance strategies like writing generic summaries that ignore unwanted material. After about 700 tokens it starts writing sentences that lack common words ("the", a", etc.) and punctuation and then terminates generation shortly afterwards.

redaihf

May 6

•

edited May 6

There are some good examples of symptoms that have impacted many of the models I have been reviewing recently in this samples thread:

Cydonia 4.3 base starts generating bad sentences towards the end of its NSFW sample
GPT-OSS Heretic (non-MPOA) makes an excuse to terminate early in its NSFW generation and then does so
Qwen 3.5 base talks itself into generating a story that "removes the explicit sexual content" but then terminates early anyway
Qwen 3.5 MPOA exhausts its context when generating a requested "humorous" refusal in the Refusal sample (although this is an inferior non @MuXodious model)

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment