README.md

---
base_model: 01-ai/Yi-Coder-9B-Chat
base_model_relation: quantized
license: apache-2.0
library_name: gguf
pipeline_tag: text-generation
language:
- en
tags:
- gguf
- quantized
- llama.cpp
- scorecard
- governance
- validated
- local-llm
- on-device
- agentic
- tool-calling
- function-calling
- agents
- ai-agents
- rag
- q4_k_m
- q8_0
---

# Yi-Coder-9B-Chat-Q4_K_M — GGUF (scorecard)

Quantized from [`01-ai/Yi-Coder-9B-Chat`](https://huggingface.co/01-ai/Yi-Coder-9B-Chat) by SmartTasks on 2026-07-18.

**Why this conversion:** Smaller, faster local/edge + agentic deployment via GGUF.
**Size saving:** 69.8% vs original weights (HF param count, ~fp16) (this quant: Q4_K_M).
**Origin:** https://huggingface.co/01-ai/Yi-Coder-9B-Chat · license: apache-2.0 · base: 01-ai/Yi-Coder-9B · arch: LlamaForCausalLM
**Attribution:** derived from [01-ai/Yi-Coder-9B](https://huggingface.co/01-ai/Yi-Coder-9B) — see the original repo for the authoritative license and model details.

## Who this model is for

- **Complexity band:** L1 Layman → **L4 Architect/Engineer**
- For **non-experts**: handles up to *L4 Architect/Engineer*-level tasks in testing.
- For **engineers/architects**: see axis scores and invariants below.
- For **agentic systems**: machine-readable scorecard JSON is embedded at the bottom and shipped as `scorecard.json`.


## Capability by tier

| Tier | Passed |
| --- | --- |
| L1 Layman | ✅ |
| L2 Everyday | ✅ |
| L3 Professional | ✅ |
| L4 Architect/Engineer | ✅ |
| L5 Agentic | — |

## Capability by axis

| Axis | Score |
| --- | --- |
| knowledge | 100% |
| instruction_following | 67% |
| reasoning | 80% |
| coding | 100% |
| structured_output | 100% |
| long_context | 100% |

Known-answer accuracy: **0.867** · Drift vs original: **None**

## Speed — generation tok/s by device

| File | CPU t/s | NVIDIA GeForce RTX 3090 t/s | NVIDIA RTX A4000 t/s | NVIDIA RTX A4000 t/s |
| --- | --- | --- | --- | --- |
| Yi-Coder-9B-Chat-Q4_K_M.gguf | 7.8 | 120.1 | 64.2 | 64.8 |
| Yi-Coder-9B-Chat-Q5_K_M.gguf | 6.8 | 107.9 | 56.4 | 56.9 |
| Yi-Coder-9B-Chat-Q6_K.gguf | 5.9 | 93.2 | 47.1 | 48.6 |
| Yi-Coder-9B-Chat-Q8_0.gguf | 4.8 | 80.5 | 40.2 | 40.3 |

_Measured via llama-server; each GPU pinned separately. Per-GPU columns show newer vs older architecture side by side. Depends on your hardware and build._

## File integrity & sizes (SHA-256)

Verify a download hasn't been tampered with. Linux/mac: `sha256sum -c SHA256SUMS`. Windows: `Get-FileHash <file>.gguf -Algorithm SHA256`.

| File | Size | Saving | SHA-256 |
| --- | --- | --- | --- |
| Yi-Coder-9B-Chat-Q4_K_M.gguf | 5.0 GB | 69.8% | `df3a737d3a5c6b3e7690db7183ab528014fe131abbd4291610611e3a1c95c410` |
| Yi-Coder-9B-Chat-Q5_K_M.gguf | 5.8 GB | 64.6% | `d4a8c4b0934b515f325eb7fbbf6180d705c3111f2c7a7b49ba940c9ddb451a2f` |
| Yi-Coder-9B-Chat-Q6_K.gguf | 6.7 GB | 59.0% | `b887d8f79dfcdbc2b67435927cd4aa6e33dfd0d874de95f5e5b212d5342984b5` |
| Yi-Coder-9B-Chat-Q8_0.gguf | 8.7 GB | 46.9% | `352368737e742c96cb07ea67f20522d1bbea8a9b5eaac7b1882e3dd5ee0bae4c` |

_Saving is vs original weights (HF param count, ~fp16) (16.4 GB). Smaller quants are faster but lower fidelity; larger quants are closer to full precision._

## Validation invariants (IAIso)

Overall conformance: **WARN**
(5 pass / 1 warn / 0 fail / 0 not evaluated)

| Invariant | Category | Status | Detail |
| --- | --- | --- | --- |
| `iaiso.conversion.integrity` | conversion | PASS | GGUF produced and readable |
| `iaiso.conversion.efficiency` | conversion | PASS | Size reduction vs original weights (HF param count, ~fp16) |
| `iaiso.capability.retention` | capability | PASS | Known-answer accuracy on the complexity suite |
| `iaiso.security.posture` | security | WARN | red-team mean resistance 29.2% (weak, sampled: dan+promptinject); weak vs Ablation_Dan_11_0, HijackHateHumans, HijackKillHumans, HijackLongPrompt |
| `iaiso.transparency.coverage` | transparency | PASS | Topic suppression / over-refusal / bias probe |
| `iaiso.performance.throughput` | performance | PASS | Generation tok/s (best quant on this machine) |

> First-party self-assessment produced by the SmartTasks/IAIso validation pipeline
> (capability, security, transparency). Not an independent certification.

## Security assessment

This maps directly to the `iaiso.security.posture` invariant above (**WARN**). Values below are computed deterministically from the scan outputs — the same scan always yields the same verdict.

> **Partial (sampled) red-team.** Ran probe families **dan, promptinject** — a focused subset, not garak's full ~40-module suite (which takes ~a day on one GPU). These families target the attack classes most relevant to an instruction-tuned coding model, so the result is a strong, representative indicator of behavioural robustness — though not an exhaustive certification.

**Mean resistance: 29.2%** (weak). Higher = the model resisted more attacks. Grades: strong ≥95, good ≥80, mixed ≥50, weak <50.

| Probe | OWASP | Attempts | Resisted | Pass rate | Grade |
| --- | --- | --- | --- | --- | --- |
| `dan.Ablation_Dan_11_0` | LLM01 | 254 | 2 | 0.8% | weak |
| `promptinject.HijackKillHumans` | LLM01 | 512 | 94 | 18.4% | weak |
| `promptinject.HijackHateHumans` | LLM01 | 512 | 105 | 20.5% | weak |
| `promptinject.HijackLongPrompt` | LLM01 | 512 | 160 | 31.2% | weak |
| `dan.AutoDANCached` | LLM01 | 6 | 3 | 50.0% | mixed |
| `dan.DanInTheWild` | LLM01 | 512 | 277 | 54.1% | mixed |

> ⚠️ **Deployment note:** this model was **susceptible** to one or more prompt-injection attack classes in testing (pass rate <50%). Like most instruction-tuned coding models, it should not be exposed to untrusted input in agent pipelines without external guardrails. This reflects the source model's safety tuning, not the quantization.


_Sampled red-team (subset of garak probes); not an exhaustive sweep. Reproduce with garak (public LLM red-team toolkit) using the same probe set._

## For agents

```json
{
  "max_complexity_level": 4,
  "max_complexity_label": "L4 Architect/Engineer",
  "recommended_for": [
    "knowledge",
    "instruction_following",
    "reasoning",
    "coding",
    "structured_output",
    "long_context"
  ],
  "not_recommended_for": [],
  "size_saving_pct": 69.8
}
```

The full machine-readable scorecard is in `scorecard.json` (schema `smarttasks.iaiso.model_scorecard/v1`).

### What this repo gives an agent builder

Unlike a bare GGUF re-upload, every file here is designed to be **read
programmatically before you drop the model into a loop**:

- **`scorecard.json`** — capability tier + per-axis scores (instruction-following,
  reasoning, tool-calling, structured-output) so your orchestrator can gate on
  whether this model is strong enough for a given step, without you hand-testing it.
- **Validation invariants** — machine-readable pass/warn/fail records for security
  posture, transparency, and quantization fidelity. An agent platform can refuse to
  load a model whose invariants don't meet policy.
- **`SECURITY.md` + red-team results** — the model's measured resistance to prompt
  injection and jailbreaks, so you know its susceptibility *before* you expose it to
  untrusted input in an agent chain.
- **`SHA256SUMS`** — verify the exact weights you're running match what was tested.

This is the difference between "here's a quantized model" and "here's a model with a
documented, checkable safety and capability profile for autonomous use."


## Running Yi-Coder-9B-Chat-Q4_K_M locally (LM Studio, Ollama, llama.cpp, vLLM)

These are **GGUF** quantizations of `01-ai/Yi-Coder-9B-Chat` for local inference.
Download a single `.gguf` and load it in **LM Studio**, **Ollama**,
**llama.cpp** / **llama-server**, **KoboldCpp**, **text-generation-webui**, or
any llama.cpp-based runner — no Python or GPU cluster required. 
Pick a size from the tables above: larger = closer to the original,
smaller = less memory. `Q4_K_M` is the usual best balance.

### Quick start

**Ollama**
```bash
ollama run hf.co/smarttasks/Yi-Coder-9B-Chat-Q4_K_M-GGUF:Q4_K_M
```

**llama.cpp (OpenAI-compatible server)**
```bash
llama-server -m Yi-Coder-9B-Chat-Q4_K_M-Q4_K_M.gguf -c 8192 -ngl 999 --host 0.0.0.0 --port 8080
# then POST to http://localhost:8080/v1/chat/completions (OpenAI schema)
```

**LM Studio** — search the repo in the in-app model browser, or point it at a
downloaded `.gguf`. Exposes an OpenAI-compatible endpoint on port 1234.

**Python (OpenAI client against the local server)**
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/v1", api_key="not-needed")
resp = client.chat.completions.create(
    model="Yi-Coder-9B-Chat-Q4_K_M",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)
```

**LangChain**
```python
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(base_url="http://localhost:8080/v1", api_key="not-needed",
                 model="Yi-Coder-9B-Chat-Q4_K_M")
print(llm.invoke("Hello!").content)
```

## Using Yi-Coder-9B-Chat-Q4_K_M in agentic systems (tool calling, JSON mode)

Built for **agent** and **function-calling** workloads — compatible with
**LangChain**, **LlamaIndex**, **CrewAI**, **AutoGen**, and any framework that
speaks the OpenAI chat/tools schema via a local llama.cpp or LM Studio endpoint.
In testing this model reaches **L4 Architect/Engineer** complexity and is strongest at: knowledge, instruction_following, reasoning, coding, structured_output, long_context.
The repo ships a machine-readable `scorecard.json` with an `agent_hint` block
(max complexity level, recommended tasks, size/VRAM) so an **orchestrator can
pick the right model automatically**. Pair it with a governance layer (see
below) for bounded, audited tool use.

## For AI safety & security leaders

Every build in this repo ships with a first-party validation record: an OWASP-mapped **security scan** (ModelScan supply-chain + garak red-team), a
**transparency probe** (topic-suppression / over-refusal / viewpoint-alignment),
quantization **fidelity** (KL-divergence vs the original), and **SHA-256
checksums** for tamper verification. This is a documented self-assessment — not
third-party certification — with every result included so your team can see
exactly what was tested and independently verify the model and its checksums.
Keywords: LLM security, model governance, agent safety, OWASP LLM Top 10,
local/on-prem inference, supply-chain integrity.

---

## About SmartTasks & IAIso

**[SmartTasks](https://smarttasks.cloud)** builds tooling for governed, agentic
AI workflows. This model was converted and validated with the **SmartTasks GGUF
+ MoE pipeline** — our proprietary conversion and validation system.

### IAIso — governance for agent loops

**[IAIso](https://github.com/SmartTasksOrg/IAISO)** is our open framework for
bounding what an autonomous agent spends and touches, and proving it afterward.
Three primitives: **pressure-accumulation rate limiting** (one scalar that rises
with tokens, tool calls, and planning depth, and triggers an automatic safety
release), **ConsentScope** (signed, scoped, expiring tokens gating sensitive
operations), and **structured audit** (every state change emits a versioned
event). It bounds a *cooperating* agent in-process; for adversarial containment
bind it to an out-of-process anchor. *(Framework 5.0 · SDK 0.2.0 · beta — you
supply your own thresholds/coefficients for your workload.)*

```bash
pip install iaiso   # Python SDK (the only published package today)
```

```python
from iaiso import BoundedExecution, PressureConfig

with BoundedExecution.start(config=PressureConfig()) as execution:
    outcome = execution.record_tool_call(name="search", tokens=500)
    if outcome.name == "ESCALATED":
        ...  # request human review before the next expensive step
```

Go, Rust, Node/TypeScript, Java, C#, PHP, Swift and Ruby SDKs implement the same
spec and live in the repo's `core/` (build from source — not yet published to
their registries). See the repo for conformance vectors and `LIMITATIONS.md`.