Instructions to use Orvyth/engrym-seed-base-9b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Orvyth/engrym-seed-base-9b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Orvyth/engrym-seed-base-9b")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Orvyth/engrym-seed-base-9b", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Orvyth/engrym-seed-base-9b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Orvyth/engrym-seed-base-9b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Orvyth/engrym-seed-base-9b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Orvyth/engrym-seed-base-9b
- SGLang
How to use Orvyth/engrym-seed-base-9b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Orvyth/engrym-seed-base-9b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Orvyth/engrym-seed-base-9b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Orvyth/engrym-seed-base-9b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Orvyth/engrym-seed-base-9b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Orvyth/engrym-seed-base-9b with Docker Model Runner:
docker model run hf.co/Orvyth/engrym-seed-base-9b
File size: 3,423 Bytes
02ebb57 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 | ---
license: other
license_name: qwen
license_link: https://huggingface.co/Qwen
base_model:
- Qwen/Qwen3.5
tags:
- agent
- tool-calling
- thinking
- function-calling
- gguf
- ollama
- long-context
pipeline_tag: text-generation
library_name: transformers
---
# Engrym Seed Base 9B
Orvyth's seed-tier brain — the recommended member of a local model family built for **tool-using
agents**. Qwen3.5 hybrid linear-attention architecture, **262,144-token native context**, first-class
tool calling.
**Weights are distributed via the Ollama registry.**
```bash
ollama run Orvyth/engrym-seed:base
```
## The ladder
| Tag | Class | Size | 77-task | Tool calls |
|---|---|---:|---:|---:|
| `:nano` | Nano 2B | 2.1 GB | 90.8/143 | 12/12 |
| `:flash` | Flash 4B | 4.6 GB | 124/143 | 12/12 |
| **`:base`** | **Base 9B** | **9.5 GB** | **131/143** | **12/12** |
| `:pro-27b-q4` | Pro 27B v2 Q4 | 16.5 GB | 134/143 | 12/12 |
| `:pro` | Pro 27B v2 Q8 | 28.6 GB | 134/143 | 12/12 |
| `:pro-e` | Pro-E 27B *(experimental)* | 28.6 GB | 137/143 | 12/12 |
## Evaluation
77 tasks · 143 points · `temperature=0` · `max_tokens=16384` · `seed=42` · one attempt · deterministic
validators · **no LLM judge**. Scores are bound to the exact published blobs.
These are **first-party** numbers. Repeated runs on an uncontended GPU are deterministic (zero spread
across n=2 for every model measured), Receipts: https://huggingface.co/datasets/Orvyth/engrym-seed-receipts
## Compute modes — the score above is a floor
Asking the model to work *deliberately* (reason step by step, verify against every constraint, then
answer) recovers points on tasks it otherwise fails. Base: **131 → 134**. The gain is largest for the
smallest models — Nano gains **+11.8**. On the small end, that is worth more than a model upgrade.
## Defaults
`temperature 0.2` · `top_p 0.9` · `top_k 20` · `num_ctx 32768` · `num_predict 8192`
Native context is 262,144; larger requests are clamped. Default is 32,768 because defaulting to the
native maximum made a 9.5 GB model request ~19 GB of RAM to start. If a prompt exceeds `num_ctx`,
Ollama returns HTTP 400 — it does not silently truncate.
## Lineage
| Stage | Provenance |
|---|---|
| Base | `Qwen/Qwen3.5` — hybrid linear-attention |
| Merge | Ornith-1.0-9B × Qwythos-9B — TIES, 0.5 / 0.5 |
| Tune | Orvyth identity + chip-calling; LoRA merged into the weights |
| Build | Converted and quantized in-house with Orvyth trainkit |
## What is in the artifact
Weights, identity, and tool-call generation. Memory, governed tool execution, safety enforcement,
adapters and multi-agent routing are Orvyth platform concerns, not part of the GGUF. Tool calling is
an output capability — the host validates, authorizes and executes.
## Limits
- Scores are first-party and single-suite. Treat small gaps between adjacent models as unresolved.
- The identity tune is light; under a heavy external system prompt behavior can defer to the base model.
- The 27B is substantially slower per tool call than the 9B. Prefer Base or Flash for agent loops.
- The MTP speculative-decoding head is not included in these builds.
- Tags are mutable — pin the digest for production and evaluations.
---
**ORVYTH** — Intelligence. Governed. *Ground truth over hype. Prove before you claim.*
|