Instructions to use MainStack/marvy-1-14B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use MainStack/marvy-1-14B-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="MainStack/marvy-1-14B-GGUF",
	filename="marvy-14B-Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use MainStack/marvy-1-14B-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf MainStack/marvy-1-14B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf MainStack/marvy-1-14B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf MainStack/marvy-1-14B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf MainStack/marvy-1-14B-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf MainStack/marvy-1-14B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf MainStack/marvy-1-14B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf MainStack/marvy-1-14B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf MainStack/marvy-1-14B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/MainStack/marvy-1-14B-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use MainStack/marvy-1-14B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "MainStack/marvy-1-14B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MainStack/marvy-1-14B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/MainStack/marvy-1-14B-GGUF:Q4_K_M

Ollama
How to use MainStack/marvy-1-14B-GGUF with Ollama:
```
ollama run hf.co/MainStack/marvy-1-14B-GGUF:Q4_K_M
```

Unsloth Studio

How to use MainStack/marvy-1-14B-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for MainStack/marvy-1-14B-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for MainStack/marvy-1-14B-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for MainStack/marvy-1-14B-GGUF to start chatting

How to use MainStack/marvy-1-14B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf MainStack/marvy-1-14B-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "MainStack/marvy-1-14B-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use MainStack/marvy-1-14B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf MainStack/marvy-1-14B-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default MainStack/marvy-1-14B-GGUF:Q4_K_M

Run Hermes

hermes

Docker Model Runner
How to use MainStack/marvy-1-14B-GGUF with Docker Model Runner:
```
docker model run hf.co/MainStack/marvy-1-14B-GGUF:Q4_K_M
```

Lemonade

How to use MainStack/marvy-1-14B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull MainStack/marvy-1-14B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.marvy-1-14B-GGUF-Q4_K_M

List all available models

lemonade list

marvy-1-14B-GGUF / USAGE.md

tgetsov

Upload USAGE.md with huggingface_hub

9b991d4 verified about 10 hours ago

preview code

raw

history blame contribute delete

7.39 kB

Using marvy-1-14B

marvy-1-14B is a ServiceNow delivery specialist. This guide covers every common way to run it — cloud or fully local — plus how to wire it into OpenCode.

Choosing a format
Recommended system prompt & settings
Transformers (PyTorch)
vLLM (OpenAI-compatible server)
MLX (Apple Silicon, native)
LM Studio (GUI + local server)
Ollama / llama.cpp (GGUF)
LoRA adapter (apply on the base)
Use marvy-1-14B in OpenCode
Prompt recipes per task

Choosing a format

You want…	Use	Repo
Max quality, GPU/server	Merged FP16	`MainStack/marvy-1-14B`
Apple Silicon, native speed	Merged (MLX)	`MainStack/marvy-1-14B`
Laptop / CPU / Ollama / LM Studio	GGUF (Q4_K_M or Q8_0)	`MainStack/marvy-1-14B-GGUF`
Smallest download, compose yourself	LoRA adapter (~175 MB)	`MainStack/marvy-1-14B-lora`

Recommended system prompt & settings

Always lead with the delivery-consultant system prompt — marvy was trained with it:

You are a senior ServiceNow delivery consultant. You produce precise, implementation-grade
artifacts: business analyses, requirements, solution design documents, user stories with
acceptance criteria, test cases, and validation reviews. You favor out-of-the-box
capabilities, cite concrete tables/plugins/sys_ids when relevant, and write in clear
professional English.

Use case	temperature	top_p	max_tokens
Structured artifacts (SDD, stories, test cases)	0.3 – 0.5	0.9	1024 – 4096
Exploratory brainstorming	0.7 – 0.9	0.95	1024
Validation / critique	0.2 – 0.4	0.9	1024 – 2048

Transformers (PyTorch)

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "MainStack/marvy-1-14B"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")

SYSTEM = "You are a senior ServiceNow delivery consultant. ..."  # full prompt above
messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": "Write a user story with acceptance criteria for P1 SLA escalation."},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=1024, temperature=0.4, top_p=0.9)
print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))

vLLM (OpenAI-compatible server)

pip install vllm
vllm serve MainStack/marvy-1-14B --served-model-name marvy-1-14B

curl -s http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "marvy-1-14B", "temperature": 0.4,
  "messages": [
    {"role":"system","content":"You are a senior ServiceNow delivery consultant. ..."},
    {"role":"user","content":"Draft the Incident Management section of an SDD."}
  ]}'

MLX (Apple Silicon, native)

pip install mlx-lm

# one-off
python -m mlx_lm generate --model MainStack/marvy-1-14B \
  --system-prompt "You are a senior ServiceNow delivery consultant. ..." \
  --prompt "Write test cases for a Major Incident workflow." --max-tokens 1024 --temp 0.4

# OpenAI-compatible server
python -m mlx_lm server --model MainStack/marvy-1-14B --port 8080

LM Studio (GUI + local server)

Install the model — either search MainStack/marvy-1-14B-GGUF in the in-app model browser, or place a local copy under ~/.lmstudio/models/MainStack/marvy-1-14B/ (MLX or GGUF layout).

Load it from the GUI, or:

lms load MainStack/marvy-1-14B
lms server start          # OpenAI-compatible on http://localhost:1234/v1

In the Chat tab, set the system prompt (above) and temperature ~0.4.

Ollama / llama.cpp (GGUF)

# Ollama — pull straight from the Hub
ollama run hf.co/MainStack/marvy-1-14B-GGUF:Q4_K_M

# llama.cpp
llama-cli -hf MainStack/marvy-1-14B-GGUF:Q4_K_M \
  -p "Write a user story with acceptance criteria for P1 SLA escalation." --temp 0.4

Quant	Size	Use when
`Q4_K_M`	~9 GB	Default — best size/quality balance
`Q8_0`	~16 GB	Highest fidelity, near-FP16

LoRA adapter (apply on the base)

# MLX
python -m mlx_lm generate --model Qwen/Qwen2.5-14B-Instruct \
  --adapter-path .  --system-prompt "You are a senior ServiceNow delivery consultant. ..." \
  --prompt "Validate this requirement and list follow-up questions: ..." --max-tokens 1024

# PEFT
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = "Qwen/Qwen2.5-14B-Instruct"
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, "MainStack/marvy-1-14B-lora")

Use marvy-1-14B in OpenCode

marvy runs behind any OpenAI-compatible endpoint (LM Studio, mlx_lm server, vLLM). Register it as a custom provider in opencode.json.

Start a local server (LM Studio shown; adjust port for others):

lms load MainStack/marvy-1-14B && lms server start   # http://localhost:1234/v1

Add the provider to your project opencode.json (or global ~/.config/opencode/opencode.json):

{
  "provider": {
    "lmstudio": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "LM Studio (local)",
      "options": { "baseURL": "http://localhost:1234/v1" },
      "models": {
        "marvy-1-14B": { "name": "marvy-1-14B (ServiceNow delivery)" }
      }
    }
  }
}

Select lmstudio/marvy-1-14B in the OpenCode model picker.

marvy-1-14B is a drafting specialist, not a tool-use/agentic fine-tune. It excels at producing delivery artifacts inside chat; for MCP tool-calling agent loops, keep a frontier model as the orchestrator and switch to marvy for drafting.

Prompt recipes per task

Task	Prompt skeleton
Business analysis	"Produce a Business Analysis for the following engagement: . Cover organization, IT landscape, scope, and risks."
Requirements	"Extract structured requirements (id, category, requirement, priority, target_phase, success_metric) from: ."
Stakeholders	"Build a stakeholder register (role, name, interest, influence, RACI) for: ."
SDD section	"Write the section of a Solution Design Document for a ServiceNow implementation. Include design decisions and concrete tables/plugins."
User story	"Write a ServiceNow user story with acceptance criteria for: ."
Implementation plan	"Given this story, describe the implementation: tables, plugins, configuration, records touched, manual follow-ups. Story: ."
Test case	"Write a test case (pre-conditions, steps, expected results, pass/fail) for the story: ."
Validation	"Validate this artifact against ServiceNow best practice and the SOW. List gaps, risks, and follow-up questions. Artifact: ."