Instructions to use ryancook/chromadb-context-1-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ryancook/chromadb-context-1-gguf with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="ryancook/chromadb-context-1-gguf",
	filename="chromadb-context-1-BF16.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use ryancook/chromadb-context-1-gguf with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf ryancook/chromadb-context-1-gguf:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf ryancook/chromadb-context-1-gguf:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf ryancook/chromadb-context-1-gguf:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf ryancook/chromadb-context-1-gguf:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf ryancook/chromadb-context-1-gguf:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf ryancook/chromadb-context-1-gguf:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf ryancook/chromadb-context-1-gguf:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf ryancook/chromadb-context-1-gguf:Q4_K_M

Use Docker

docker model run hf.co/ryancook/chromadb-context-1-gguf:Q4_K_M

LM Studio
Jan

vLLM

How to use ryancook/chromadb-context-1-gguf with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ryancook/chromadb-context-1-gguf"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ryancook/chromadb-context-1-gguf",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ryancook/chromadb-context-1-gguf:Q4_K_M

Ollama
How to use ryancook/chromadb-context-1-gguf with Ollama:
```
ollama run hf.co/ryancook/chromadb-context-1-gguf:Q4_K_M
```

Unsloth Studio new

How to use ryancook/chromadb-context-1-gguf with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for ryancook/chromadb-context-1-gguf to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for ryancook/chromadb-context-1-gguf to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for ryancook/chromadb-context-1-gguf to start chatting

Pi new

How to use ryancook/chromadb-context-1-gguf with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf ryancook/chromadb-context-1-gguf:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "ryancook/chromadb-context-1-gguf:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use ryancook/chromadb-context-1-gguf with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf ryancook/chromadb-context-1-gguf:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default ryancook/chromadb-context-1-gguf:Q4_K_M

Run Hermes

hermes

Docker Model Runner
How to use ryancook/chromadb-context-1-gguf with Docker Model Runner:
```
docker model run hf.co/ryancook/chromadb-context-1-gguf:Q4_K_M
```

Lemonade

How to use ryancook/chromadb-context-1-gguf with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull ryancook/chromadb-context-1-gguf:Q4_K_M

Run and chat with the model

lemonade run user.chromadb-context-1-gguf-Q4_K_M

List all available models

lemonade list

Chroma Context-1 — GGUF (llama.cpp)

GGUF weights for Chroma Context-1, converted for llama.cpp and any runtime that loads GGUF (LM Studio, Ollama with compatible import paths, local servers, etc.).

This repository exists because the upstream model is distributed in PyTorch / safetensors form only. These files are the same weights in GGUF, with a range of llama-quantize presets so you can trade quality for VRAM and disk.

Upstream (source of truth)

	Link
Original weights & model card	`chromadb/context-1`
Architecture family	gpt-oss MoE (see upstream card; base traceable to OpenAI `gpt-oss-20b`)
License	Apache 2.0 (unchanged; you must comply with upstream terms)

Attribution: All tensors are derived from chromadb/context-1. This repo is a community conversion and is not affiliated with or endorsed by Chroma. For behavior, safety, and intended use, read the official model card first.

Quick start

1. Install a recent llama.cpp build (or use a GUI that bundles it).

2. Download this repository:

huggingface-cli download ryancook/chromadb-context-1-gguf --local-dir ./chromadb-context-1-gguf

3. Run (example — adjust paths and context length to your hardware):

llama-cli -m ./chromadb-context-1-gguf/chromadb-context-1-Q4_0.gguf -cnv --color -ngl 99

Swap the filename for any published chromadb-context-1-*.gguf from the Files tab (for example Q4_K_M or MXFP4_MOE when available).

Choosing a file

Start here (good defaults for most people):

Priority	File pattern	When to use
1	`…-Q4_K_M.gguf` or `…-Q5_K_M.gguf`	Best general-purpose balance of quality and size (if present in this repo).
2	`…-MXFP4_MOE.gguf`	Smaller MoE-oriented layout; strong choice when supported by your llama.cpp build/GPU stack.
3	`…-Q4_0.gguf` / `…-Q5_0.gguf`	Simpler legacy-style quants; predictable tradeoffs.
4	`…-bf16.gguf`	Full BF16 fidelity (~40 GiB class); for reference or maximum quality when you have RAM/VRAM.

Other presets (IQ*, TQ*, Q2_K, Q3_K*, Q6_K, Q8_0, F16, …) may appear in the Files tab as they are published. Lower-bit and ternary formats are experimental for quality; profile on your workload before relying on them.

Tip: The Files and versions view on Hugging Face is authoritative for what is available in each commit. Filenames follow chromadb-context-1-<PRESET>.gguf.

Conversion pipeline

Reproducible high-level steps:

Obtain weights from chromadb/context-1 (Apache 2.0).
Convert to GGUF with llama.cpp convert_hf_to_gguf.py (BF16 output from upstream bf16 checkpoint).
Quantize with llama-quantize using the preset named in each filename (Q4_0, Q4_K_M, MXFP4_MOE, etc.).

Reproducibility

Conversions for this collection were produced with ggml-org/llama.cpp at commit 07ba6d275 (short SHA; matches upstream convert_hf_to_gguf.py / llama-quantize from that tree). Newer llama.cpp revisions are generally backward compatible for GGUF loading, but you may see small numerical differences if you re-quantize.

Hardware & context

VRAM / RAM: MoE models route only a subset of experts per token; still treat published sizes as a guide and monitor peak usage at your target context length.
Context length: Upstream supports a very long context window; practical limits depend on KV cache size and quant. Start with a smaller -c / context setting and increase only after you confirm stability.

License

Same as upstream: Apache 2.0. Keep chromadb/context-1 attribution visible when you redistribute or ship products built on these files.

Model tree for ryancook/chromadb-context-1-gguf

Base model

openai/gpt-oss-20b

Finetuned

chromadb/context-1

Quantized

(7)

this model

ryancook
/

chromadb-context-1-gguf