Instructions to use inflatebot/MN-12B-Mag-Mell-R1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use inflatebot/MN-12B-Mag-Mell-R1 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="inflatebot/MN-12B-Mag-Mell-R1") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("inflatebot/MN-12B-Mag-Mell-R1") model = AutoModelForCausalLM.from_pretrained("inflatebot/MN-12B-Mag-Mell-R1", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use inflatebot/MN-12B-Mag-Mell-R1 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "inflatebot/MN-12B-Mag-Mell-R1" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "inflatebot/MN-12B-Mag-Mell-R1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/inflatebot/MN-12B-Mag-Mell-R1
- SGLang
How to use inflatebot/MN-12B-Mag-Mell-R1 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "inflatebot/MN-12B-Mag-Mell-R1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "inflatebot/MN-12B-Mag-Mell-R1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "inflatebot/MN-12B-Mag-Mell-R1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "inflatebot/MN-12B-Mag-Mell-R1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use inflatebot/MN-12B-Mag-Mell-R1 with Docker Model Runner:
docker model run hf.co/inflatebot/MN-12B-Mag-Mell-R1
Permission to use Mag Mell in a free game (donations + Patreon early access)
EDIT (update and apology)
Hi inflatebot, and thanks for Mag Mell. It's great for roleplay, and I read your answers in discussion #17 about the license.
I make free games that use Mag Mell locally, running on the player's own machine. Before asking, I should have checked the license properly: I assumed Mag Mell was Apache 2.0, like most of the models it's merged from, and only later noticed the page has no license field. I'm sorry for asking after the fact rather than before, and I want to be upfront about where things stand.
What's already out there. I've published one game, Academia Silvermoor, an adult (18+) visual novel with a chat mode, where Mag Mell R1 (bartowski's IQ4_XS GGUF) generates the characters' replies locally on the player's PC. Nothing is sent to any server.
- itch.io (free, optional voluntary donations): https://grognar1861.itch.io/academia-silvermoor
- Patreon (early access to later chapters and fan art; each chapter is released for free afterwards): https://www.patreon.com/cw/Grognar1861
- I also plan to post it on F95zone, free.
The first release (0.1.0, one day) did include the GGUF inside the download, which is what my question 2 below was about. I've already replaced it: the current version (0.1.1) does not include the model. It downloads it on first run straight from the Hugging Face repo, after showing the player the model's name, author and link and asking for confirmation. The old version is no longer downloadable.
Credits. You, the model link and the upstream authors (Mistral-Nemo-Base-2407, Chronos-Gold, Sunrose, Bophades, Wissenschaft, Gutenberg v4, Magnum v2.5 KTO), plus bartowski for the quantization, are credited in the game and on the store pages.
What I'm asking (I understand you can't speak for Sunfall or Sunrose, so only about your part):
- Are you OK with this kind of use: a free game with voluntary donations and a Patreon for early story chapters?
- Are you OK with the model being downloaded by the player from Hugging Face rather than bundled? (That's how it works now.)
- If you're OK with it but would like a condition, such as a particular credit wording or a link, I'll add it.
If you're not comfortable with it, just say so. I'll take the game down and look for an alternative model with a clearer license; no hard feelings. A short answer here would help me a lot, and if you're fine with it, saying so publicly in this thread would give me a record. Thanks!
I don't mind that at all! Absolutely no complaints from me. Mag Mell is in a weird spot license-wise, but as far as I'm aware, Apache 2.0 is semi-viral, so I'm not sure if any of the license overrides from other people apply. It's always been "spiritually Apache" in my heart though.
The only thing I ask, in addition to credit which you're already doing, is that if you make any significant money from it, you consider joining Allura's Discord server and contributing to mutual aid there from time to time. Not to say you have to, I'm not about to demand payment, just that you consider it, after you succeed. :3
That's all, godspeed o7
Thank you so much for the quick and kind reply, and for being so generous about it. Honestly, projects like mine wouldn’t get off the ground without people like you who build these models and share them openly.
I also want to say that Mag Mell really is something special. I tried several different models while developing the game, and yours stood out clearly from the rest. The quality of its Spanish in particular is very good, which matters a lot for a game that is played entirely in conversation. It’s a big part of why the characters feel alive.
I’ll keep the Discord suggestion in mind, and I truly hope the project does well enough that I can give something back someday. Thanks again for your work, and for the model.
Spanish, that's really interesting!!
I swear I gotta send some SAE probes into this model or something, because the amount of stuff people tell me that's completely unexpected is wild. Nemo was already pretty special, a lot of people came together to hack on it, and I just happened to put the right pieces together in the right order. I'm just as gobsmacked as others when I hear the continued praise.
(A bit of cosmic irony is that I wish I could make something this good but on a newer model that supports multimodal input, LMAO. My personal use case of AI has centered around that and not having it is a huge drag!)
(A bit of cosmic irony is that I wish I could make something this good but on a newer model that supports multimodal input, LMAO. My personal use case of AI has centered around that and not having it is a huge drag!)
I have made attempts to graft in Mistral Multi-Modal Vision... It's not a dead end road. However I have one task ahead of me first. Will keep in touch.
I got close last time. Mag mell with a Pixtral Vision tower. IT DID SEE, however the tokenizer and weights got merged incorrectly due to my lack of knowledge a few months back. It still output coherent responses but the grammar and punctuation was absolutely all over the place. But it did, describe images I showed to it. I'll be in touch.