Instructions to use MarinaraSpaghetti/NemoMix-Unleashed-12B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use MarinaraSpaghetti/NemoMix-Unleashed-12B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="MarinaraSpaghetti/NemoMix-Unleashed-12B")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("MarinaraSpaghetti/NemoMix-Unleashed-12B") model = AutoModelForCausalLM.from_pretrained("MarinaraSpaghetti/NemoMix-Unleashed-12B") - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use MarinaraSpaghetti/NemoMix-Unleashed-12B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "MarinaraSpaghetti/NemoMix-Unleashed-12B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MarinaraSpaghetti/NemoMix-Unleashed-12B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/MarinaraSpaghetti/NemoMix-Unleashed-12B
- SGLang
How to use MarinaraSpaghetti/NemoMix-Unleashed-12B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "MarinaraSpaghetti/NemoMix-Unleashed-12B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MarinaraSpaghetti/NemoMix-Unleashed-12B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "MarinaraSpaghetti/NemoMix-Unleashed-12B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MarinaraSpaghetti/NemoMix-Unleashed-12B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use MarinaraSpaghetti/NemoMix-Unleashed-12B with Docker Model Runner:
docker model run hf.co/MarinaraSpaghetti/NemoMix-Unleashed-12B
My settings
After gathering stuff from the discussion thread for this model and the SillyTavern discord server, I settled on this:
Text completion preset
Story string
Instruct
Lorebook for all chats
I really like the responses I get with this, so I am sharing it here.
Thanks for sharing!
I have a question so please, explain it to me because I might be missunderstanding how those work in the end or not... I do not know. Why there's often a STORY STRING instead of CONTEXT TEMPLATE in those settings that people upload? It's just a CONTEXT TEMPLATE preset, right? Why is it not called CONTEXT TEMPLATE and INSTRUCT PRESET the way it's actually called in Silly Tavern? Is it not the same and I am using those wrong? Judging on format of those and how they load into Silly Tavern, they seem to be context templates with a weird name? If so - why call them like that instead of a context template? Please, help me understand it once and for all :-( Thx in advance!
Hey @sphiratrioth666 ,hope this helps a bit!
Yeah, it helps! Thank you very much! I was already importing those and using them without problems - it's just that I did not pay attention to those small, grey headlines so I did not understand why story string instead of context template when I import it "under" context template :-D Now it's clear, haha. Thx a lot!
Looks like the host for the website is down for those configs. π
Any chance you could re-upload them?
Looks like the host for the website is down for those configs. π
Any chance you could re-upload them?
The links work fine still for me, try from a different browser, maybe?
Looks like the host for the website is down for those configs. π
Any chance you could re-upload them?
The links work fine still for me, try from a different browser, maybe?
Thanks for checking on your end!
Turns out it was Malwarebytes blocking the domain lol
Thanks for checking on your end!
Turns out it was Malwarebytes blocking the domain lol
Glad it worked!
Hey @sphiratrioth666 ,hope this helps a bit!
Yeah, it helps! Thank you very much! I was already importing those and using them without problems - it's just that I did not pay attention to those small, grey headlines so I did not understand why story string instead of context template when I import it "under" context template :-D Now it's clear, haha. Thx a lot!
What kind of program are you using? Thank you in advance for your reply.
What kind of program are you using? Thank you in advance for your reply.
SillyTavern is the frontend I use.
With the new ST update that split it from Sampler, Instruct, and Context settings... Now we have Sampler, Instruct, Context, and System Prompt "areas". This broke a bunch of my settings, curious what I should be putting in system prompt?
With the new ST update that split it from Sampler, Instruct, and Context settings... Now we have Sampler, Instruct, Context, and System Prompt "areas". This broke a bunch of my settings, curious what I should be putting in system prompt?
They had a good idea with the 'master import', but the split to different sections is confusing and super unintuitive. System Prompt is just the βYou are {{char}}, please respond in the role-play [β¦]β part, so basically, main instruction.

