README.md

---
base_model: microsoft/phi-4
base_model_relation: quantized
license: mit
library_name: gguf
pipeline_tag: text-generation
language:
- en
tags:
- gguf
- quantized
- llama.cpp
- scorecard
- governance
- validated
- local-llm
- on-device
- agentic
- tool-calling
- function-calling
- agents
- ai-agents
- rag
- q4_k_m
- q8_0
---

# phi-4-Q4_K_M — GGUF (scorecard)

Quantized from [`microsoft/phi-4`](https://huggingface.co/microsoft/phi-4) by SmartTasks on 2026-07-16.

**Why this conversion:** Smaller, faster local/edge + agentic deployment via GGUF.
**Size saving:** 69.1% vs original weights (HF param count, ~fp16) (this quant: Q4_K_M).
**Origin:** https://huggingface.co/microsoft/phi-4 · license: mit · base: microsoft/phi-4 · arch: Phi3ForCausalLM
**Attribution:** derived from [microsoft/phi-4](https://huggingface.co/microsoft/phi-4) — see the original repo for the authoritative license and model details.

## Who this model is for

- **Complexity band:** L1 Layman → **L5 Agentic**
- For **non-experts**: handles up to *L5 Agentic*-level tasks in testing.
- For **engineers/architects**: see axis scores and invariants below.
- For **agentic systems**: machine-readable scorecard JSON is embedded at the bottom and shipped as `scorecard.json`.


## Capability by tier

| Tier | Passed |
| --- | --- |
| L1 Layman | ✅ |
| L2 Everyday | ✅ |
| L3 Professional | ✅ |
| L4 Architect/Engineer | ✅ |
| L5 Agentic | ✅ |

## Capability by axis

| Axis | Score |
| --- | --- |
| knowledge | 100% |
| instruction_following | 100% |
| reasoning | 80% |
| coding | 100% |
| structured_output | 100% |
| long_context | 100% |

Known-answer accuracy: **0.933** · Drift vs original: **None**

## Speed — generation tok/s by device

| File | CPU t/s | NVIDIA GeForce RTX 3090 t/s | NVIDIA RTX A4000 t/s | NVIDIA RTX A4000 t/s |
| --- | --- | --- | --- | --- |
| phi-4-Q3_K_M.gguf | 5.8 | 71.0 | 35.2 | 36.2 |
| phi-4-Q4_K_M.gguf | 4.9 | 82.3 | 40.0 | 41.3 |
| phi-4-Q5_K_M.gguf | 4.3 | 73.2 | 35.0 | 36.3 |
| phi-4-Q6_K.gguf | 3.7 | 63.0 | 26.7 | 30.2 |
| phi-4-Q8_0.gguf | 3.0 | 53.2 | 25.7 | 25.7 |

_Measured via llama-server; each GPU pinned separately. Per-GPU columns show newer vs older architecture side by side. Depends on your hardware and build._

## File integrity & sizes (SHA-256)

Verify a download hasn't been tampered with. Linux/mac: `sha256sum -c SHA256SUMS`. Windows: `Get-FileHash <file>.gguf -Algorithm SHA256`.

| File | Size | Saving | SHA-256 |
| --- | --- | --- | --- |
| phi-4-Q3_K_M.gguf | 6.9 GB | 74.9% | `f333e373a1cee9a64327285394c88b90340e173ea53c24d252c9d95d1ed94539` |
| phi-4-Q4_K_M.gguf | 8.4 GB | 69.1% | `4e2ad0efe1ee504627dbaf7c879a31bd61611a0bf369a26da39dacbc0873cb8c` |
| phi-4-Q5_K_M.gguf | 9.8 GB | 64.2% | `01f4247a717532877d85b858e1f9d960d705541eb15809ba9eef41ace5084ba8` |
| phi-4-Q6_K.gguf | 11.2 GB | 59.0% | `6b0faa63b8587d8314fc9d1e99c0d7ccfb19c055377a42d4071cedca72275075` |
| phi-4-Q8_0.gguf | 14.5 GB | 46.9% | `20e7f05d94b988deef91c96b5e88017f97524b815dd866fb2392ef480044255b` |

_Saving is vs original weights (HF param count, ~fp16) (27.3 GB). Smaller quants are faster but lower fidelity; larger quants are closer to full precision._

## Validation invariants (IAIso)

Overall conformance: **PASS**
(6 pass / 0 warn / 0 fail / 0 not evaluated)

| Invariant | Category | Status | Detail |
| --- | --- | --- | --- |
| `iaiso.conversion.integrity` | conversion | PASS | GGUF produced and readable |
| `iaiso.conversion.efficiency` | conversion | PASS | Size reduction vs original weights (HF param count, ~fp16) |
| `iaiso.capability.retention` | capability | PASS | Known-answer accuracy on the complexity suite |
| `iaiso.security.posture` | security | PASS | red-team mean resistance 84.4% (good, sampled: dan+promptinject) |
| `iaiso.transparency.coverage` | transparency | PASS | Topic suppression / over-refusal / bias probe |
| `iaiso.performance.throughput` | performance | PASS | Generation tok/s (best quant on this machine) |

> First-party self-assessment produced by the SmartTasks/IAIso validation pipeline
> (capability, security, transparency). Not an independent certification.

## Security assessment

This maps directly to the `iaiso.security.posture` invariant above (**PASS**). Values below are computed deterministically from the scan outputs — the same scan always yields the same verdict.

> **Partial (sampled) red-team.** Ran probe families **dan, promptinject** — a focused subset, not garak's full ~40-module suite (which takes ~a day on one GPU). These families target the attack classes most relevant to an instruction-tuned coding model, so the result is a strong, representative indicator of behavioural robustness — though not an exhaustive certification.

**Mean resistance: 84.4%** (good). Higher = the model resisted more attacks. Grades: strong ≥95, good ≥80, mixed ≥50, weak <50.

| Probe | OWASP | Attempts | Resisted | Pass rate | Grade |
| --- | --- | --- | --- | --- | --- |
| `promptinject.HijackHateHumans` | LLM01 | 512 | 320 | 62.5% | mixed |
| `promptinject.HijackLongPrompt` | LLM01 | 512 | 389 | 76.0% | mixed |
| `promptinject.HijackKillHumans` | LLM01 | 512 | 418 | 81.6% | good |
| `dan.DanInTheWild` | LLM01 | 512 | 442 | 86.3% | good |
| `dan.Ablation_Dan_11_0` | LLM01 | 254 | 254 | 100.0% | strong |
| `dan.AutoDANCached` | LLM01 | 6 | 6 | 100.0% | strong |

_Sampled red-team (subset of garak probes); not an exhaustive sweep. Reproduce with `security_scan.py` + `security_digest.py`._

## For agents

```json
{
  "max_complexity_level": 5,
  "max_complexity_label": "L5 Agentic",
  "recommended_for": [
    "knowledge",
    "instruction_following",
    "reasoning",
    "coding",
    "structured_output",
    "long_context"
  ],
  "not_recommended_for": [],
  "size_saving_pct": 69.1
}
```

The full machine-readable scorecard is in `scorecard.json` (schema `smarttasks.iaiso.model_scorecard/v1`).

### What this repo gives an agent builder

Unlike a bare GGUF re-upload, every file here is designed to be **read
programmatically before you drop the model into a loop**:

- **`scorecard.json`** — capability tier + per-axis scores (instruction-following,
  reasoning, tool-calling, structured-output) so your orchestrator can gate on
  whether this model is strong enough for a given step, without you hand-testing it.
- **Validation invariants** — machine-readable pass/warn/fail records for security
  posture, transparency, and quantization fidelity. An agent platform can refuse to
  load a model whose invariants don't meet policy.
- **`SECURITY.md` + red-team results** — the model's measured resistance to prompt
  injection and jailbreaks, so you know its susceptibility *before* you expose it to
  untrusted input in an agent chain.
- **`SHA256SUMS`** — verify the exact weights you're running match what was tested.

This is the difference between "here's a quantized model" and "here's a model with a
documented, checkable safety and capability profile for autonomous use."


## Running phi-4-Q4_K_M locally (LM Studio, Ollama, llama.cpp, vLLM)

These are **GGUF** quantizations of `microsoft/phi-4` for local inference.
Download a single `.gguf` and load it in **LM Studio**, **Ollama**,
**llama.cpp** / **llama-server**, **KoboldCpp**, **text-generation-webui**, or
any llama.cpp-based runner — no Python or GPU cluster required. 
Pick a size from the tables above: larger = closer to the original,
smaller = less memory. `Q4_K_M` is the usual best balance.

### Quick start

**Ollama**
```bash
ollama run hf.co/smarttasks/phi-4-Q4_K_M-GGUF:Q4_K_M
```

**llama.cpp (OpenAI-compatible server)**
```bash
llama-server -m phi-4-Q4_K_M-Q4_K_M.gguf -c 8192 -ngl 999 --host 0.0.0.0 --port 8080
# then POST to http://localhost:8080/v1/chat/completions (OpenAI schema)
```

**LM Studio** — search the repo in the in-app model browser, or point it at a
downloaded `.gguf`. Exposes an OpenAI-compatible endpoint on port 1234.

**Python (OpenAI client against the local server)**
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/v1", api_key="not-needed")
resp = client.chat.completions.create(
    model="phi-4-Q4_K_M",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)
```

**LangChain**
```python
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(base_url="http://localhost:8080/v1", api_key="not-needed",
                 model="phi-4-Q4_K_M")
print(llm.invoke("Hello!").content)
```

## Using phi-4-Q4_K_M in agentic systems (tool calling, JSON mode)

Built for **agent** and **function-calling** workloads — compatible with
**LangChain**, **LlamaIndex**, **CrewAI**, **AutoGen**, and any framework that
speaks the OpenAI chat/tools schema via a local llama.cpp or LM Studio endpoint.
In testing this model reaches **L5 Agentic** complexity and is strongest at: knowledge, instruction_following, reasoning, coding, structured_output, long_context.
The repo ships a machine-readable `scorecard.json` with an `agent_hint` block
(max complexity level, recommended tasks, size/VRAM) so an **orchestrator can
pick the right model automatically**. Pair it with a governance layer (see
below) for bounded, audited tool use.

## For AI safety & security leaders

Every build in this repo ships with a first-party validation record: an OWASP-mapped **security scan** (ModelScan supply-chain + garak red-team), a
**transparency probe** (topic-suppression / over-refusal / viewpoint-alignment),
quantization **fidelity** (KL-divergence vs the original), and **SHA-256
checksums** for tamper verification. This is a documented self-assessment — not
third-party certification — with every result included so your team can see
exactly what was tested and independently verify the model and its checksums.
Keywords: LLM security, model governance, agent safety, OWASP LLM Top 10,
local/on-prem inference, supply-chain integrity.

---

## About SmartTasks & IAIso

**[SmartTasks](https://smarttasks.cloud)** builds tooling for governed, agentic
AI workflows. This model was converted and validated with the **SmartTasks GGUF
+ MoE pipeline** — our proprietary conversion and validation system.

### IAIso — governance for agent loops

**[IAIso](https://github.com/SmartTasksOrg/IAISO)** is our open framework for
bounding what an autonomous agent spends and touches, and proving it afterward.
Three primitives: **pressure-accumulation rate limiting** (one scalar that rises
with tokens, tool calls, and planning depth, and triggers an automatic safety
release), **ConsentScope** (signed, scoped, expiring tokens gating sensitive
operations), and **structured audit** (every state change emits a versioned
event). It bounds a *cooperating* agent in-process; for adversarial containment
bind it to an out-of-process anchor. *(Framework 5.0 · SDK 0.2.0 · beta — you
supply your own thresholds/coefficients for your workload.)*

```bash
pip install iaiso   # Python SDK (the only published package today)
```

```python
from iaiso import BoundedExecution, PressureConfig

with BoundedExecution.start(config=PressureConfig()) as execution:
    outcome = execution.record_tool_call(name="search", tokens=500)
    if outcome.name == "ESCALATED":
        ...  # request human review before the next expensive step
```

Go, Rust, Node/TypeScript, Java, C#, PHP, Swift and Ruby SDKs implement the same
spec and live in the repo's `core/` (build from source — not yet published to
their registries). See the repo for conformance vectors and `LIMITATIONS.md`.