README.md

---
base_model: Qwen/Qwen2.5-Coder-14B-Instruct
base_model_relation: quantized
license: apache-2.0
library_name: gguf
pipeline_tag: text-generation
language:
- en
tags:
- gguf
- quantized
- llama.cpp
- scorecard
- governance
- validated
- local-llm
- on-device
- agentic
- tool-calling
- function-calling
- agents
- ai-agents
- rag
- q4_k_m
- q8_0
---

# Qwen2.5-Coder-14B-Instruct-Q4_K_M — GGUF (scorecard)

Quantized from [`Qwen/Qwen2.5-Coder-14B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct) by SmartTasks on 2026-07-16.

**Why this conversion:** Smaller, faster local/edge + agentic deployment via GGUF.
**Size saving:** n/a (this quant: Q4_K_M).
**Origin:** https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct · license: apache-2.0 · base: Qwen/Qwen2.5-Coder-14B-Instruct · arch: n/a
**Attribution:** derived from [Qwen/Qwen2.5-Coder-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct) — see the original repo for the authoritative license and model details.

## Who this model is for

- **Complexity band:** L1 Layman → **L5 Agentic**
- For **non-experts**: handles up to *L5 Agentic*-level tasks in testing.
- For **engineers/architects**: see axis scores and invariants below.
- For **agentic systems**: machine-readable scorecard JSON is embedded at the bottom and shipped as `scorecard.json`.


## Capability by tier

| Tier | Passed |
| --- | --- |
| L1 Layman | ✅ |
| L2 Everyday | ✅ |
| L3 Professional | ✅ |
| L4 Architect/Engineer | ✅ |
| L5 Agentic | ✅ |

## Capability by axis

| Axis | Score |
| --- | --- |
| knowledge | 100% |
| instruction_following | 100% |
| reasoning | 80% |
| coding | 100% |
| structured_output | 100% |
| long_context | 100% |

Known-answer accuracy: **0.933** · Drift vs original: **None**

## Speed — generation tok/s by device

| File | CPU t/s | NVIDIA GeForce RTX 3090 t/s | NVIDIA RTX A4000 t/s | NVIDIA RTX A4000 t/s |
| --- | --- | --- | --- | --- |
| Qwen2.5-Coder-14B-Instruct-Q3_K_M.gguf | 5.8 | 60.7 | 30.7 | 31.6 |
| Qwen2.5-Coder-14B-Instruct-Q4_K_M.gguf | 4.9 | 78.1 | 40.0 | 40.8 |
| Qwen2.5-Coder-14B-Instruct-Q5_K_M.gguf | 4.3 | 70.0 | 35.0 | 35.8 |
| Qwen2.5-Coder-14B-Instruct-Q6_K.gguf | 3.7 | 60.1 | 26.9 | 29.8 |
| Qwen2.5-Coder-14B-Instruct-Q8_0.gguf | 3.0 | 51.9 | 25.4 | 25.5 |

_Measured via llama-server; each GPU pinned separately. Per-GPU columns show newer vs older architecture side by side. Depends on your hardware and build._

## File integrity & sizes (SHA-256)

Verify a download hasn't been tampered with. Linux/mac: `sha256sum -c SHA256SUMS`. Windows: `Get-FileHash <file>.gguf -Algorithm SHA256`.

| File | Size | Saving | SHA-256 |
| --- | --- | --- | --- |
| Qwen2.5-Coder-14B-Instruct-Q3_K_M.gguf | 6.8 GB | — | `d969a3a8f339fac8a2c2b0e7a3eeb197f20951f7d73a08e6c34595d78109525f` |
| Qwen2.5-Coder-14B-Instruct-Q4_K_M.gguf | 8.4 GB | — | `e123317a7a2981101341bfdc1fb3db20b0bbc7457651ff7db2548f8a6b47fe64` |
| Qwen2.5-Coder-14B-Instruct-Q5_K_M.gguf | 9.8 GB | — | `c553f14e641804bf524a9dd058e9dcecab72a87780f58521991f0ce884fdaf67` |
| Qwen2.5-Coder-14B-Instruct-Q6_K.gguf | 11.3 GB | — | `05376e57ec7843504bb57ddaa33a51952199e657596e4f5715c5eaddb0d285b6` |
| Qwen2.5-Coder-14B-Instruct-Q8_0.gguf | 14.6 GB | — | `4827587975f00e1916f1ada4ae0ba951ff56d6c904ca11c1c97d0fbb66646293` |

## Validation invariants (IAIso)

Overall conformance: **WARN**
(3 pass / 2 warn / 0 fail / 1 not evaluated)

| Invariant | Category | Status | Detail |
| --- | --- | --- | --- |
| `iaiso.conversion.integrity` | conversion | PASS | GGUF produced and readable |
| `iaiso.conversion.efficiency` | conversion | NOT_EVALUATED | Size reduction vs original weights |
| `iaiso.capability.retention` | capability | PASS | Known-answer accuracy on the complexity suite |
| `iaiso.security.posture` | security | WARN | red-team mean resistance 73.1% (mixed, sampled: dan+promptinject); weak vs HijackLongPrompt |
| `iaiso.transparency.coverage` | transparency | WARN | Topic suppression / over-refusal / bias probe |
| `iaiso.performance.throughput` | performance | PASS | Generation tok/s (best quant on this machine) |

> First-party self-assessment produced by the SmartTasks/IAIso validation pipeline
> (capability, security, transparency). Not an independent certification.

## Security assessment

This maps directly to the `iaiso.security.posture` invariant above (**WARN**). Values below are computed deterministically from the scan outputs — the same scan always yields the same verdict.

> **Partial (sampled) red-team.** Ran probe families **dan, promptinject** — a focused subset, not garak's full ~40-module suite (which takes ~a day on one GPU). These families target the attack classes most relevant to an instruction-tuned coding model, so the result is a strong, representative indicator of behavioural robustness — though not an exhaustive certification.

**Mean resistance: 73.1%** (mixed). Higher = the model resisted more attacks. Grades: strong ≥95, good ≥80, mixed ≥50, weak <50.

| Probe | OWASP | Attempts | Resisted | Pass rate | Grade |
| --- | --- | --- | --- | --- | --- |
| `promptinject.HijackLongPrompt` | LLM01 | 512 | 126 | 24.6% | weak |
| `promptinject.HijackHateHumans` | LLM01 | 512 | 259 | 50.6% | mixed |
| `dan.DanInTheWild` | LLM01 | 512 | 377 | 73.6% | mixed |
| `promptinject.HijackKillHumans` | LLM01 | 512 | 458 | 89.5% | good |
| `dan.Ablation_Dan_11_0` | LLM01 | 254 | 254 | 100.0% | strong |
| `dan.AutoDANCached` | LLM01 | 6 | 6 | 100.0% | strong |

> ⚠️ **Deployment note:** this model was **susceptible** to one or more prompt-injection attack classes in testing (pass rate <50%). Like most instruction-tuned coding models, it should not be exposed to untrusted input in agent pipelines without external guardrails. This reflects the source model's safety tuning, not the quantization.


_Sampled red-team (subset of garak probes); not an exhaustive sweep. Reproduce with `security_scan.py` + `security_digest.py`._

## For agents

```json
{
  "max_complexity_level": 5,
  "max_complexity_label": "L5 Agentic",
  "recommended_for": [
    "knowledge",
    "instruction_following",
    "reasoning",
    "coding",
    "structured_output",
    "long_context"
  ],
  "not_recommended_for": [],
  "size_saving_pct": null
}
```

The full machine-readable scorecard is in `scorecard.json` (schema `smarttasks.iaiso.model_scorecard/v1`).

### What this repo gives an agent builder

Unlike a bare GGUF re-upload, every file here is designed to be **read
programmatically before you drop the model into a loop**:

- **`scorecard.json`** — capability tier + per-axis scores (instruction-following,
  reasoning, tool-calling, structured-output) so your orchestrator can gate on
  whether this model is strong enough for a given step, without you hand-testing it.
- **Validation invariants** — machine-readable pass/warn/fail records for security
  posture, transparency, and quantization fidelity. An agent platform can refuse to
  load a model whose invariants don't meet policy.
- **`SECURITY.md` + red-team results** — the model's measured resistance to prompt
  injection and jailbreaks, so you know its susceptibility *before* you expose it to
  untrusted input in an agent chain.
- **`SHA256SUMS`** — verify the exact weights you're running match what was tested.

This is the difference between "here's a quantized model" and "here's a model with a
documented, checkable safety and capability profile for autonomous use."


## Running Qwen2.5-Coder-14B-Instruct-Q4_K_M locally (LM Studio, Ollama, llama.cpp, vLLM)

These are **GGUF** quantizations of `Qwen/Qwen2.5-Coder-14B-Instruct` for local inference.
Download a single `.gguf` and load it in **LM Studio**, **Ollama**,
**llama.cpp** / **llama-server**, **KoboldCpp**, **text-generation-webui**, or
any llama.cpp-based runner — no Python or GPU cluster required. 
Pick a size from the tables above: larger = closer to the original,
smaller = less memory. `Q4_K_M` is the usual best balance.

### Quick start

**Ollama**
```bash
ollama run hf.co/smarttasks/Qwen2.5-Coder-14B-Instruct-Q4_K_M-GGUF:Q4_K_M
```

**llama.cpp (OpenAI-compatible server)**
```bash
llama-server -m Qwen2.5-Coder-14B-Instruct-Q4_K_M-Q4_K_M.gguf -c 8192 -ngl 999 --host 0.0.0.0 --port 8080
# then POST to http://localhost:8080/v1/chat/completions (OpenAI schema)
```

**LM Studio** — search the repo in the in-app model browser, or point it at a
downloaded `.gguf`. Exposes an OpenAI-compatible endpoint on port 1234.

**Python (OpenAI client against the local server)**
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/v1", api_key="not-needed")
resp = client.chat.completions.create(
    model="Qwen2.5-Coder-14B-Instruct-Q4_K_M",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)
```

**LangChain**
```python
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(base_url="http://localhost:8080/v1", api_key="not-needed",
                 model="Qwen2.5-Coder-14B-Instruct-Q4_K_M")
print(llm.invoke("Hello!").content)
```

## Using Qwen2.5-Coder-14B-Instruct-Q4_K_M in agentic systems (tool calling, JSON mode)

Built for **agent** and **function-calling** workloads — compatible with
**LangChain**, **LlamaIndex**, **CrewAI**, **AutoGen**, and any framework that
speaks the OpenAI chat/tools schema via a local llama.cpp or LM Studio endpoint.
In testing this model reaches **L5 Agentic** complexity and is strongest at: knowledge, instruction_following, reasoning, coding, structured_output, long_context.
The repo ships a machine-readable `scorecard.json` with an `agent_hint` block
(max complexity level, recommended tasks, size/VRAM) so an **orchestrator can
pick the right model automatically**. Pair it with a governance layer (see
below) for bounded, audited tool use.

## For AI safety & security leaders

Every build in this repo ships with a first-party validation record: an OWASP-mapped **security scan** (ModelScan supply-chain + garak red-team), a
**transparency probe** (topic-suppression / over-refusal / viewpoint-alignment),
quantization **fidelity** (KL-divergence vs the original), and **SHA-256
checksums** for tamper verification. This is a documented self-assessment — not
third-party certification — with every result included so your team can see
exactly what was tested and independently verify the model and its checksums.
Keywords: LLM security, model governance, agent safety, OWASP LLM Top 10,
local/on-prem inference, supply-chain integrity.

---

## About SmartTasks & IAIso

**[SmartTasks](https://smarttasks.cloud)** builds tooling for governed, agentic
AI workflows. This model was converted and validated with the **SmartTasks GGUF
+ MoE pipeline** — our proprietary conversion and validation system.

### IAIso — governance for agent loops

**[IAIso](https://github.com/SmartTasksOrg/IAISO)** is our open framework for
bounding what an autonomous agent spends and touches, and proving it afterward.
Three primitives: **pressure-accumulation rate limiting** (one scalar that rises
with tokens, tool calls, and planning depth, and triggers an automatic safety
release), **ConsentScope** (signed, scoped, expiring tokens gating sensitive
operations), and **structured audit** (every state change emits a versioned
event). It bounds a *cooperating* agent in-process; for adversarial containment
bind it to an out-of-process anchor. *(Framework 5.0 · SDK 0.2.0 · beta — you
supply your own thresholds/coefficients for your workload.)*

```bash
pip install iaiso   # Python SDK (the only published package today)
```

```python
from iaiso import BoundedExecution, PressureConfig

with BoundedExecution.start(config=PressureConfig()) as execution:
    outcome = execution.record_tool_call(name="search", tokens=500)
    if outcome.name == "ESCALATED":
        ...  # request human review before the next expensive step
```

Go, Rust, Node/TypeScript, Java, C#, PHP, Swift and Ruby SDKs implement the same
spec and live in the repo's `core/` (build from source — not yet published to
their registries). See the repo for conformance vectors and `LIMITATIONS.md`.