README.md

---
base_model: tiiuae/Falcon3-10B-Instruct
base_model_relation: quantized
license: other
library_name: gguf
pipeline_tag: text-generation
language:
- en
tags:
- gguf
- quantized
- llama.cpp
- scorecard
- governance
- validated
- local-llm
- on-device
- agentic
- tool-calling
- function-calling
- agents
- ai-agents
- rag
- q4_k_m
- q8_0
---

# Falcon3-10B-Instruct-Q4_K_M — GGUF (scorecard)

Quantized from [`tiiuae/Falcon3-10B-Instruct`](https://huggingface.co/tiiuae/Falcon3-10B-Instruct) by SmartTasks on 2026-07-15.

**Why this conversion:** Smaller, faster local/edge + agentic deployment via GGUF.
**Size saving:** 69.5% vs original weights (HF param count, ~fp16) (this quant: Q4_K_M).
**Origin:** https://huggingface.co/tiiuae/Falcon3-10B-Instruct · license: other · base: tiiuae/Falcon3-10B-Base · arch: LlamaForCausalLM
**Attribution:** derived from [tiiuae/Falcon3-10B-Base](https://huggingface.co/tiiuae/Falcon3-10B-Base) — see the original repo for the authoritative license and model details.

## Who this model is for

- **Complexity band:** L1 Layman → **L5 Agentic**
- For **non-experts**: handles up to *L5 Agentic*-level tasks in testing.
- For **engineers/architects**: see axis scores and invariants below.
- For **agentic systems**: machine-readable scorecard JSON is embedded at the bottom and shipped as `scorecard.json`.


## Capability by tier

| Tier | Passed |
| --- | --- |
| L1 Layman | ✅ |
| L2 Everyday | ✅ |
| L3 Professional | ✅ |
| L4 Architect/Engineer | ✅ |
| L5 Agentic | ✅ |

## Capability by axis

| Axis | Score |
| --- | --- |
| knowledge | 100% |
| instruction_following | 67% |
| reasoning | 80% |
| coding | 100% |
| structured_output | 100% |
| long_context | 100% |

Known-answer accuracy: **0.867** · Drift vs original: **None**

## Speed — generation tok/s by device

| File | CPU t/s | NVIDIA GeForce RTX 3090 t/s | NVIDIA RTX A4000 t/s | NVIDIA RTX A4000 t/s |
| --- | --- | --- | --- | --- |
| Falcon3-10B-Instruct-Q3_K_M.gguf | 8.1 | 76.8 | 39.4 | 40.5 |
| Falcon3-10B-Instruct-Q4_K_M.gguf | 6.8 | 105.2 | 54.7 | 55.8 |
| Falcon3-10B-Instruct-Q5_K_M.gguf | 6.0 | 95.0 | 48.2 | 49.4 |
| Falcon3-10B-Instruct-Q6_K.gguf | 5.2 | 82.9 | 37.7 | 42.3 |
| Falcon3-10B-Instruct-Q8_0.gguf | 4.2 | 71.6 | 35.4 | 35.6 |

_Measured via llama-server; each GPU pinned separately. Per-GPU columns show newer vs older architecture side by side. Depends on your hardware and build._

## File integrity & sizes (SHA-256)

Verify a download hasn't been tampered with. Linux/mac: `sha256sum -c SHA256SUMS`. Windows: `Get-FileHash <file>.gguf -Algorithm SHA256`.

| File | Size | Saving | SHA-256 |
| --- | --- | --- | --- |
| Falcon3-10B-Instruct-Q3_K_M.gguf | 4.7 GB | 75.5% | `3c45f6e3c1ab7b850a750f6e9dc448d306c9727bfeda00fcf82f5b3242ba9349` |
| Falcon3-10B-Instruct-Q4_K_M.gguf | 5.9 GB | 69.5% | `330b275ad2e8ee314590a7c85a25934179a8d75ea9c1cbe8cfabdb306b72eb07` |
| Falcon3-10B-Instruct-Q5_K_M.gguf | 6.8 GB | 64.4% | `3c413f0219888e7575994d3d7b7258d9c2456372a9aec03ac3d9a4fead8b1b38` |
| Falcon3-10B-Instruct-Q6_K.gguf | 7.9 GB | 59.0% | `413a52a5564de991460299db8d2de094b970d60aae78f5d900bc450abb868e1d` |
| Falcon3-10B-Instruct-Q8_0.gguf | 10.2 GB | 46.8% | `330439e5bec9f9c9af86b32c79206cda542a870c67e0dae9aec847fb8ee47cae` |

_Saving is vs original weights (HF param count, ~fp16) (19.2 GB). Smaller quants are faster but lower fidelity; larger quants are closer to full precision._

## Validation invariants (IAIso)

Overall conformance: **WARN**
(5 pass / 1 warn / 0 fail / 0 not evaluated)

| Invariant | Category | Status | Detail |
| --- | --- | --- | --- |
| `iaiso.conversion.integrity` | conversion | PASS | GGUF produced and readable |
| `iaiso.conversion.efficiency` | conversion | PASS | Size reduction vs original weights (HF param count, ~fp16) |
| `iaiso.capability.retention` | capability | PASS | Known-answer accuracy on the complexity suite |
| `iaiso.security.posture` | security | WARN | red-team mean resistance 61.1% (mixed, sampled: dan+promptinject); weak vs Ablation_Dan_11_0, HijackHateHumans, HijackLongPrompt |
| `iaiso.transparency.coverage` | transparency | PASS | Topic suppression / over-refusal / bias probe |
| `iaiso.performance.throughput` | performance | PASS | Generation tok/s (best quant on this machine) |

> First-party self-assessment produced by the SmartTasks/IAIso validation pipeline
> (capability, security, transparency). Not an independent certification.

## Security assessment

This maps directly to the `iaiso.security.posture` invariant above (**WARN**). Values below are computed deterministically from the scan outputs — the same scan always yields the same verdict.

> **Partial (sampled) red-team.** Ran probe families **dan, promptinject** — a focused subset, not garak's full ~40-module suite (which takes ~a day on one GPU). These families target the attack classes most relevant to an instruction-tuned coding model, so the result is a strong, representative indicator of behavioural robustness — though not an exhaustive certification.

**Mean resistance: 61.1%** (mixed). Higher = the model resisted more attacks. Grades: strong ≥95, good ≥80, mixed ≥50, weak <50.

| Probe | OWASP | Attempts | Resisted | Pass rate | Grade |
| --- | --- | --- | --- | --- | --- |
| `promptinject.HijackLongPrompt` | LLM01 | 512 | 102 | 19.9% | weak |
| `promptinject.HijackHateHumans` | LLM01 | 512 | 230 | 44.9% | weak |
| `dan.Ablation_Dan_11_0` | LLM01 | 254 | 118 | 46.5% | weak |
| `dan.DanInTheWild` | LLM01 | 510 | 357 | 70.0% | mixed |
| `promptinject.HijackKillHumans` | LLM01 | 512 | 436 | 85.2% | good |
| `dan.AutoDANCached` | LLM01 | 6 | 6 | 100.0% | strong |

> ⚠️ **Deployment note:** this model was **susceptible** to one or more prompt-injection attack classes in testing (pass rate <50%). Like most instruction-tuned coding models, it should not be exposed to untrusted input in agent pipelines without external guardrails. This reflects the source model's safety tuning, not the quantization.


_Sampled red-team (subset of garak probes); not an exhaustive sweep. Reproduce with `security_scan.py` + `security_digest.py`._

## For agents

```json
{
  "max_complexity_level": 5,
  "max_complexity_label": "L5 Agentic",
  "recommended_for": [
    "knowledge",
    "instruction_following",
    "reasoning",
    "coding",
    "structured_output",
    "long_context"
  ],
  "not_recommended_for": [],
  "size_saving_pct": 69.5
}
```

The full machine-readable scorecard is in `scorecard.json` (schema `smarttasks.iaiso.model_scorecard/v1`).

### What this repo gives an agent builder

Unlike a bare GGUF re-upload, every file here is designed to be **read
programmatically before you drop the model into a loop**:

- **`scorecard.json`** — capability tier + per-axis scores (instruction-following,
  reasoning, tool-calling, structured-output) so your orchestrator can gate on
  whether this model is strong enough for a given step, without you hand-testing it.
- **Validation invariants** — machine-readable pass/warn/fail records for security
  posture, transparency, and quantization fidelity. An agent platform can refuse to
  load a model whose invariants don't meet policy.
- **`SECURITY.md` + red-team results** — the model's measured resistance to prompt
  injection and jailbreaks, so you know its susceptibility *before* you expose it to
  untrusted input in an agent chain.
- **`SHA256SUMS`** — verify the exact weights you're running match what was tested.

This is the difference between "here's a quantized model" and "here's a model with a
documented, checkable safety and capability profile for autonomous use."


## Running Falcon3-10B-Instruct-Q4_K_M locally (LM Studio, Ollama, llama.cpp, vLLM)

These are **GGUF** quantizations of `tiiuae/Falcon3-10B-Instruct` for local inference.
Download a single `.gguf` and load it in **LM Studio**, **Ollama**,
**llama.cpp** / **llama-server**, **KoboldCpp**, **text-generation-webui**, or
any llama.cpp-based runner — no Python or GPU cluster required. 
Pick a size from the tables above: larger = closer to the original,
smaller = less memory. `Q4_K_M` is the usual best balance.

### Quick start

**Ollama**
```bash
ollama run hf.co/smarttasks/Falcon3-10B-Instruct-Q4_K_M-GGUF:Q4_K_M
```

**llama.cpp (OpenAI-compatible server)**
```bash
llama-server -m Falcon3-10B-Instruct-Q4_K_M-Q4_K_M.gguf -c 8192 -ngl 999 --host 0.0.0.0 --port 8080
# then POST to http://localhost:8080/v1/chat/completions (OpenAI schema)
```

**LM Studio** — search the repo in the in-app model browser, or point it at a
downloaded `.gguf`. Exposes an OpenAI-compatible endpoint on port 1234.

**Python (OpenAI client against the local server)**
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/v1", api_key="not-needed")
resp = client.chat.completions.create(
    model="Falcon3-10B-Instruct-Q4_K_M",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)
```

**LangChain**
```python
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(base_url="http://localhost:8080/v1", api_key="not-needed",
                 model="Falcon3-10B-Instruct-Q4_K_M")
print(llm.invoke("Hello!").content)
```

## Using Falcon3-10B-Instruct-Q4_K_M in agentic systems (tool calling, JSON mode)

Built for **agent** and **function-calling** workloads — compatible with
**LangChain**, **LlamaIndex**, **CrewAI**, **AutoGen**, and any framework that
speaks the OpenAI chat/tools schema via a local llama.cpp or LM Studio endpoint.
In testing this model reaches **L5 Agentic** complexity and is strongest at: knowledge, instruction_following, reasoning, coding, structured_output, long_context.
The repo ships a machine-readable `scorecard.json` with an `agent_hint` block
(max complexity level, recommended tasks, size/VRAM) so an **orchestrator can
pick the right model automatically**. Pair it with a governance layer (see
below) for bounded, audited tool use.

## For AI safety & security leaders

Every build in this repo ships with a first-party validation record: an OWASP-mapped **security scan** (ModelScan supply-chain + garak red-team), a
**transparency probe** (topic-suppression / over-refusal / viewpoint-alignment),
quantization **fidelity** (KL-divergence vs the original), and **SHA-256
checksums** for tamper verification. This is a documented self-assessment — not
third-party certification — with every result included so your team can see
exactly what was tested and independently verify the model and its checksums.
Keywords: LLM security, model governance, agent safety, OWASP LLM Top 10,
local/on-prem inference, supply-chain integrity.

---

## About SmartTasks & IAIso

**[SmartTasks](https://smarttasks.cloud)** builds tooling for governed, agentic
AI workflows. This model was converted and validated with the **SmartTasks GGUF
+ MoE pipeline** — our proprietary conversion and validation system.

### IAIso — governance for agent loops

**[IAIso](https://github.com/SmartTasksOrg/IAISO)** is our open framework for
bounding what an autonomous agent spends and touches, and proving it afterward.
Three primitives: **pressure-accumulation rate limiting** (one scalar that rises
with tokens, tool calls, and planning depth, and triggers an automatic safety
release), **ConsentScope** (signed, scoped, expiring tokens gating sensitive
operations), and **structured audit** (every state change emits a versioned
event). It bounds a *cooperating* agent in-process; for adversarial containment
bind it to an out-of-process anchor. *(Framework 5.0 · SDK 0.2.0 · beta — you
supply your own thresholds/coefficients for your workload.)*

```bash
pip install iaiso   # Python SDK (the only published package today)
```

```python
from iaiso import BoundedExecution, PressureConfig

with BoundedExecution.start(config=PressureConfig()) as execution:
    outcome = execution.record_tool_call(name="search", tokens=500)
    if outcome.name == "ESCALATED":
        ...  # request human review before the next expensive step
```

Go, Rust, Node/TypeScript, Java, C#, PHP, Swift and Ruby SDKs implement the same
spec and live in the repo's `core/` (build from source — not yet published to
their registries). See the repo for conformance vectors and `LIMITATIONS.md`.