Instructions to use inflatebot/MN-12B-Mag-Mell-R1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use inflatebot/MN-12B-Mag-Mell-R1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="inflatebot/MN-12B-Mag-Mell-R1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("inflatebot/MN-12B-Mag-Mell-R1")
model = AutoModelForCausalLM.from_pretrained("inflatebot/MN-12B-Mag-Mell-R1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use inflatebot/MN-12B-Mag-Mell-R1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "inflatebot/MN-12B-Mag-Mell-R1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "inflatebot/MN-12B-Mag-Mell-R1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/inflatebot/MN-12B-Mag-Mell-R1

SGLang

How to use inflatebot/MN-12B-Mag-Mell-R1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "inflatebot/MN-12B-Mag-Mell-R1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "inflatebot/MN-12B-Mag-Mell-R1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "inflatebot/MN-12B-Mag-Mell-R1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "inflatebot/MN-12B-Mag-Mell-R1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use inflatebot/MN-12B-Mag-Mell-R1 with Docker Model Runner:
```
docker model run hf.co/inflatebot/MN-12B-Mag-Mell-R1
```

This models deserves praises

#13

by GhostGate - opened Jan 9, 2025

Discussion

GhostGate

Jan 9, 2025

Hi! Everyone said so far how much they loved your model. I can also safely say that your model is one of the best I have ever tried. It crushes so many others right off the bat. But the level of character details and the realism of their portrayal is far beyond anything OG Nemo, and even larger models, are capable of. Spatial awareness might lack sometimes (not too bad if you ask me, I correct some mistakes with a nifty system prompt and an author notes at chat level), but the emotional depth the characters get it's something else. What surprises me is that it's really good at instruction following, reading between the lines and maintaining said personality in all situations. There is no "tomorrow will be better" type of attitude, or the sudden "I was very sad and depressed, but all of a sudden I am the most confident person ever". It keeps the initial attitude throughout the scenario, which is the main point. They don't fall into despair, or adapt a sultry/chipper/confident trait out of the blue, they stay within their boundaries.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment