Upload USAGE.md with huggingface_hub

USAGE.md

@@ -0,0 +1,191 @@
+# Using marvy-14B
+
+marvy-14B is a ServiceNow delivery specialist. This guide covers every common
+way to run it — cloud or fully local — plus how to wire it into OpenCode.
+
+- [Choosing a format](#choosing-a-format)
+- [Recommended system prompt & settings](#recommended-system-prompt--settings)
+- [Transformers (PyTorch)](#transformers-pytorch)
+- [vLLM (OpenAI-compatible server)](#vllm-openai-compatible-server)
+- [MLX (Apple Silicon, native)](#mlx-apple-silicon-native)
+- [LM Studio (GUI + local server)](#lm-studio-gui--local-server)
+- [Ollama / llama.cpp (GGUF)](#ollama--llamacpp-gguf)
+- [LoRA adapter (apply on the base)](#lora-adapter-apply-on-the-base)
+- [Use marvy-14B in OpenCode](#use-marvy-14b-in-opencode)
+- [Prompt recipes per task](#prompt-recipes-per-task)
+
+---
+
+## Choosing a format
+
+| You want… | Use | Repo |
+|---|---|---|
+| Max quality, GPU/server | Merged FP16 | `MainStack/marvy-14B` |
+| Apple Silicon, native speed | Merged (MLX) | `MainStack/marvy-14B` |
+| Laptop / CPU / Ollama / LM Studio | GGUF (Q4_K_M or Q8_0) | `MainStack/marvy-14B-GGUF` |
+| Smallest download, compose yourself | LoRA adapter (~175 MB) | `MainStack/marvy-14B-lora` |
+
+---
+
+## Recommended system prompt & settings
+
+Always lead with the delivery-consultant system prompt — marvy was trained with it:
+
+```
+You are a senior ServiceNow delivery consultant. You produce precise, implementation-grade
+artifacts: business analyses, requirements, solution design documents, user stories with
+acceptance criteria, test cases, and validation reviews. You favor out-of-the-box
+capabilities, cite concrete tables/plugins/sys_ids when relevant, and write in clear
+professional English.
+```
+
+| Use case | temperature | top_p | max_tokens |
+|---|---|---|---|
+| Structured artifacts (SDD, stories, test cases) | 0.3 – 0.5 | 0.9 | 1024 – 4096 |
+| Exploratory brainstorming | 0.7 – 0.9 | 0.95 | 1024 |
+| Validation / critique | 0.2 – 0.4 | 0.9 | 1024 – 2048 |
+
+---
+
+## Transformers (PyTorch)
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+model_id = "MainStack/marvy-14B"
+tok = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")
+
+SYSTEM = "You are a senior ServiceNow delivery consultant. ..."  # full prompt above
+messages = [
+    {"role": "system", "content": SYSTEM},
+    {"role": "user", "content": "Write a user story with acceptance criteria for P1 SLA escalation."},
+]
+inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
+out = model.generate(inputs, max_new_tokens=1024, temperature=0.4, top_p=0.9)
+print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))
+```
+
+## vLLM (OpenAI-compatible server)
+
+```bash
+pip install vllm
+vllm serve MainStack/marvy-14B --served-model-name marvy-14B
+```
+
+```bash
+curl -s http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
+  "model": "marvy-14B", "temperature": 0.4,
+  "messages": [
+    {"role":"system","content":"You are a senior ServiceNow delivery consultant. ..."},
+    {"role":"user","content":"Draft the Incident Management section of an SDD."}
+  ]}'
+```
+
+## MLX (Apple Silicon, native)
+
+```bash
+pip install mlx-lm
+
+# one-off
+python -m mlx_lm generate --model MainStack/marvy-14B \
+  --system-prompt "You are a senior ServiceNow delivery consultant. ..." \
+  --prompt "Write test cases for a Major Incident workflow." --max-tokens 1024 --temp 0.4
+
+# OpenAI-compatible server
+python -m mlx_lm server --model MainStack/marvy-14B --port 8080
+```
+
+## LM Studio (GUI + local server)
+
+1. **Install the model** — either search `MainStack/marvy-14B-GGUF` in the
+   in-app model browser, or place a local copy under
+   `~/.lmstudio/models/MainStack/marvy-14B/` (MLX or GGUF layout).
+2. **Load** it from the GUI, or:
+   ```bash
+   lms load MainStack/marvy-14B
+   lms server start          # OpenAI-compatible on http://localhost:1234/v1
+   ```
+3. In the Chat tab, set the system prompt (above) and temperature ~0.4.
+
+## Ollama / llama.cpp (GGUF)
+
+```bash
+# Ollama — pull straight from the Hub
+ollama run hf.co/MainStack/marvy-14B-GGUF:Q4_K_M
+
+# llama.cpp
+llama-cli -hf MainStack/marvy-14B-GGUF:Q4_K_M \
+  -p "Write a user story with acceptance criteria for P1 SLA escalation." --temp 0.4
+```
+
+| Quant | Size | Use when |
+|---|---|---|
+| `Q4_K_M` | ~9 GB | Default — best size/quality balance |
+| `Q8_0` | ~16 GB | Highest fidelity, near-FP16 |
+
+## LoRA adapter (apply on the base)
+
+```bash
+# MLX
+python -m mlx_lm generate --model Qwen/Qwen2.5-14B-Instruct \
+  --adapter-path .  --system-prompt "You are a senior ServiceNow delivery consultant. ..." \
+  --prompt "Validate this requirement and list follow-up questions: ..." --max-tokens 1024
+```
+
+```python
+# PEFT
+from peft import PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer
+base = "Qwen/Qwen2.5-14B-Instruct"
+model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="auto", device_map="auto")
+model = PeftModel.from_pretrained(model, "MainStack/marvy-14B-lora")
+```
+
+---
+
+## Use marvy-14B in OpenCode
+
+marvy runs behind any OpenAI-compatible endpoint (LM Studio, mlx_lm server,
+vLLM). Register it as a custom provider in `opencode.json`.
+
+1. **Start a local server** (LM Studio shown; adjust port for others):
+   ```bash
+   lms load MainStack/marvy-14B && lms server start   # http://localhost:1234/v1
+   ```
+2. **Add the provider** to your project `opencode.json` (or global
+   `~/.config/opencode/opencode.json`):
+   ```jsonc
+   {
+     "provider": {
+       "lmstudio": {
+         "npm": "@ai-sdk/openai-compatible",
+         "name": "LM Studio (local)",
+         "options": { "baseURL": "http://localhost:1234/v1" },
+         "models": {
+           "marvy-14B": { "name": "marvy-14B (ServiceNow delivery)" }
+         }
+       }
+     }
+   }
+   ```
+3. **Select** `lmstudio/marvy-14B` in the OpenCode model picker.
+
+> marvy-14B is a drafting specialist, not a tool-use/agentic fine-tune. It excels
+> at producing delivery artifacts inside chat; for MCP tool-calling agent loops,
+> keep a frontier model as the orchestrator and switch to marvy for drafting.
+
+---
+
+## Prompt recipes per task
+
+| Task | Prompt skeleton |
+|---|---|
+| Business analysis | "Produce a Business Analysis for the following engagement: <context>. Cover organization, IT landscape, scope, and risks." |
+| Requirements | "Extract structured requirements (id, category, requirement, priority, target_phase, success_metric) from: <notes>." |
+| Stakeholders | "Build a stakeholder register (role, name, interest, influence, RACI) for: <context>." |
+| SDD section | "Write the <section> section of a Solution Design Document for a ServiceNow <module> implementation. Include design decisions and concrete tables/plugins." |
+| User story | "Write a ServiceNow user story with acceptance criteria for: <capability>." |
+| Implementation plan | "Given this story, describe the implementation: tables, plugins, configuration, records touched, manual follow-ups. Story: <story>." |
+| Test case | "Write a test case (pre-conditions, steps, expected results, pass/fail) for the story: <story>." |
+| Validation | "Validate this artifact against ServiceNow best practice and the SOW. List gaps, risks, and follow-up questions. Artifact: <artifact>." |