README.md

---
title: PDF Injection Detector (MiMo-7B)
emoji: 🔍
colorFrom: yellow
colorTo: gray
sdk: gradio
sdk_version: 5.50.0
app_file: app.py
pinned: false
license: mit
short_description: MiMo-7B finds payloads hidden inside PDF files
tags:
  - cybersecurity
  - pdf
  - prompt-injection
  - malware
  - gradio
  - scikit-learn
  - gradient-boosting
  - embeddings
  - recommender
---

# PDF Injection Detector — MiMo-7B

Upload a PDF. The app renders it to text with the extractor that built the project corpus, ranks the
regions carrying structural signatures, asks **MiMo-7B-RL** whether a payload is hidden in the batch
you choose, and then asks a **family-naming model** which of twelve attacks it is — so the
remediation advice it prints is attached to a measured answer rather than to a guess.

It is a coursework artefact built on a synthetic corpus of 1,100 PDFs carrying harmless
EICAR/AMTSO/WICAR/RANSIM test markers. **It is not a general malware scanner.**

---

## Table of Contents

1. [The pipeline](#the-pipeline)
2. [Part 1 — The family-naming model](#part-1--the-family-naming-model)
   - [Why it exists](#why-it-exists)
   - [The four signals](#the-four-signals)
   - [Results](#results)
   - [Why 0.994 is the least interesting number](#why-0994-is-the-least-interesting-number)
   - [Per-family behaviour](#per-family-behaviour)
3. [Part 2 — The app built around it](#part-2--the-app-built-around-it)
   - [How to use it](#how-to-use-it)
   - [The controls](#the-controls)
   - [The five tabs](#the-five-tabs)
   - [Where the plan panel's numbers come from](#where-the-plan-panels-numbers-come-from)
4. [Part 3 — The models and the numbers](#part-3--the-models-and-the-numbers)
   - [Why MiMo, when Gemma scored higher](#why-mimo-when-gemma-scored-higher)
   - [One runtime](#one-runtime)
   - [Triage and the sweep](#triage-and-the-sweep)
5. [Part 4 — Provenance, and the files](#part-4--provenance-and-the-files)

---

## The pipeline

| Stage | Module | What it answers | What it is worth |
|---|---|---|---|
| 1. Extract | `corpus_text.py` | PDF bytes → text skeleton → ranked candidate regions | byte-identical to the corpus build |
| 2. Detect | `mimo.py` | *is there a payload in this region?* | F1 **0.945** (constant baseline 0.900) |
| 3. Retrieve | `neighbours.py` | *what does this region resemble?* | precision@5 **35.6%** (random 6.8%) |
| 4. Name | `family_model.py` | *which of thirteen classes is it?* | P@1 **0.994** / **0.691** — see below |

Stages 2 and 4 are separate because the two questions have very different difficulty. MiMo is good
at the first and poor at the second, and the advice the app prints depends on the second.

---

# Part 1 — The family-naming model

Built first, in [`App_Creation.ipynb`](App_Creation.ipynb) Part 2, and shipped as
`family_naming_model.pkl` (915 KB).

## Why it exists

The evaluation stage of this project measured four open language models on this corpus and found two
very different competences:

| Question | MiMo-7B-RL |
|---|---|
| Is a payload present? | **F1 0.945** — against 0.900 for flagging every file unread |
| Which family is it? | **43.3%** correct |

The second row is the one that reaches a user. This app attaches **remediation advice** to a family,
so a wrong family means someone is shown the fix for an attack their file does not contain. At 43%
that would have happened more often than not, which is why the family is not MiMo's to name.

## The four signals

The model is a fusion of four sources of evidence about the same document, none adequate alone:

| Signal | Columns | Where it comes from | What it knows | What it misses |
|---|---|---|---|---|
| Structural signatures | 12 | `corpus_text.detect_markers` on the skeleton | exactly which of twelve known payload shapes are present | anything outside those twelve |
| MiMo's own answer | 15 | the scan's family guess, verdict and parse flag | a language model's reading of the text | wrong on family 57% of the time |
| Document shape | 20 | 17 EDA measurements — entropy, object counts, streams, pages, sizes — plus 3 log companions | carrier-level anomalies | little about the payload itself |
| Neighbour vote | 13 | weighted vote of the 20 nearest corpus files in the embedding index | what the file *resembles*, including novel wording | weak alone: precision@5 35.6% |

That is 60 features. The classifier is a `HistGradientBoostingClassifier` — a few hundred small
decision trees, each correcting the errors of the ones before it. CPU-only, under a second per
prediction, no neural network anywhere in it.

## Results

Everything is **out of fold**: the 1,100 documents are split into five folds and each is predicted by
a model that never saw it in training.

| Model | P@1 | P@3 | MRR | macro-F1 |
|---|---|---|---|---|
| **gradient boosting (shipped)** | **0.994** | 0.999 | 0.996 | 0.995 |
| GB without MiMo's guess | 0.993 | 0.999 | 0.996 | 0.994 |
| logistic regression | 0.992 | 0.998 | 0.995 | 0.992 |
| GB without signatures | 0.691 | 0.823 | 0.778 | 0.666 |
| regex signatures alone | 0.669 | 0.818 | 0.758 | 0.774 |
| neighbours + shape only | 0.532 | 0.689 | 0.649 | 0.481 |
| embedding neighbours alone | 0.527 | 0.705 | 0.651 | 0.478 |
| MiMo's family guess alone | 0.498 | 0.585 | 0.597 | 0.441 |
| always answer `none` (floor) | 0.182 | 0.330 | 0.336 | 0.024 |

![Each signal alone versus the fusion](figures/fig1_signals.png)

*Chart 1 — no single signal is close. The best one alone is the regex at 0.669; the two learned
signals are worse than that. The fusion is doing real work rather than laundering one strong
feature.*

The floor matters: 18.2% of the corpus is clean, so a model that ignores the file entirely and always
answers `none` scores 0.182. Every row above is read against that, not against zero.

## Why 0.994 is the least interesting number

A model scoring 0.994 on a coursework corpus should be treated as a bug report until proven
otherwise. Removing one signal block at a time and re-running the whole procedure says where the
score comes from.

![Ablation by feature block](figures/fig2_ablation.png)

*Chart 2 — the dotted line is the full model. Removing MiMo costs 0.001. Removing the structural
signatures costs 0.303.*

**The signatures are carrying almost everything, and that is a fact about the corpus rather than a
flaw in the model.** The label *is* the inserted marker: a document is labelled
`ransomware_simulation` **because** the generator wrote a `RANSIM TEST` string into it, and the
signature block reads that string back out.

So there are two honest figures, describing two different situations:

| | P@1 | Describes |
|---|---|---|
| with signatures | **0.994** | a file carrying one of the twelve payload shapes this project generated |
| without signatures | **0.691** | a file carrying something else — the harder, realistic case |

**The app reports whichever applies to the file in front of it.** `family_model.predict` checks
whether the upload carries a known signature and quotes 0.994 or 0.691 next to its answer, instead
of printing the flattering number in both cases.

A second result worth stating: **MiMo's opinion is nearly redundant.** The model reaches 0.993
without ever being told what the language model thought — its contribution is almost entirely
recoverable from the other three signals.

## Per-family behaviour

The ablation is an average, and averages hide the shape of a failure.

![Per-family recall](figures/fig3_per_family.png)

*Chart 3 — blue is the full model, orange is the same model without signature evidence.*

`steganographic_payload`, `uri_redirect_phishing` and `dde_template_injection` survive the loss well.
**`ssrf` collapses to 0.35**, the worst of the thirteen: an SSRF payload is a URL pointing at a cloud
metadata address, so it adds no distinctive object to the PDF and reads like ordinary text.

![Confusion matrix without signatures](figures/fig4_confusion.png)

*Chart 4 — where the honest (no-signature) model's mistakes actually go.*

The confusions are not random. `ransomware_simulation` is mistaken for `javascript_injection` 15
times and `object_action_injection` for `shellcode_embedded_exe` 13 times — exactly the pairs the
generator gives a shared JavaScript launcher to. It is confusing documents that genuinely do
resemble each other.

The `none` row is the reassuring one: **190 of 200 clean files are still called clean** with no
signature evidence at all. The model is far better at *is this clean?* than at *which of twelve
attacks is this?* — the same division of competence MiMo shows, arrived at independently.

This is the argument for the ranked short-list. When the top pick is wrong the correct family is
usually still in the list — P@3 is 0.823 even in the harder regime — so the interface shows three
candidates with confidences rather than asserting one.

---

# Part 2 — The app built around it

## How to use it

Every screenshot below is of this Space as it actually runs — one real session on the
`ransomware simulation` example, captured in order, nothing mocked up or edited.

### 1. Open the Space

![The interface on load](docs/screenshots/01_landing.png)

The page comes up before anything is downloaded. The header states what a region costs — about
**4.2 s**. Read the disclaimer: the verdict, the family and the treatment are all fallible, and each
carries its own measured accuracy rather than one vague shared one.

### 2. Give it a PDF

Drag a file onto the **PDF** box, or press one of the **Examples** buttons down the left edge — one
per attack family, plus a clean control. An example drops its file into the same uploader, so it is
triaged by exactly the same code path as your own upload.

![An example loaded, with the plan underneath](docs/screenshots/02_example_loaded.png)

The uploader shows `Example_5.pdf`, not the family name. The button is labelled; the file is not.
Nothing about the document announces its own answer, and a filename never reaches a model.

### 3. Read the plan, then set the controls

Under the button the app says what it found and what it intends to read *before* any model runs.

![The controls](docs/screenshots/03_controls.png)

## The controls

| Control | What it does | When to change it |
|---|---|---|
| **Regions per batch** | How many regions one run of the model reads. It only re-cuts the same list, so a lower number means more batches, not less work. | Leave it at the maximum. A run reserves the same GPU time whatever this is set to, so lowering it inspects less of the file for the same quota. |
| **Batch to check** | Which batch this run spends. Each label states how many regions, whether they are marker or sweep, where in the skeleton they sit, and the estimated time. | Batch 1 is the most signature-dense, so start there. |
| **Sweep the rest of the document too** | On, the batches tile the whole skeleton. Off, only signature regions are read — the exact shape the evaluation measured. | Off to stay strictly inside the measured configuration; on for coverage of text no signature points at. |
| **Name the attack family (and show nearest corpus files)** | Runs stages 3 and 4 after the scan. Costs a one-off 550 MB embedding-model download. | Turn it off to save the download — but the family then falls back to MiMo's own guess, right 43% of the time instead of the model's 99.4%/69.1%. |

### 4. Press **Check this batch**

The button locks for the duration and any previous result is cleared, so nothing on screen can
belong to a different file. One press is one run over one batch — press it again on a different batch
to spend another run; the app never scans the whole file behind your back.

![A scan in progress](docs/screenshots/05_scanning.png)

*Mid-run: the button is disabled, the previous result has already been cleared, and the progress line
names what is happening — `MiMo reading 2 region(s)`. The first scan after a restart also fetches the
weights, which is the slow part.*

**The result below is worth reading carefully, because it is the case this whole design exists for.**

![The report, with the family named](docs/screenshots/06_report_family.png)

MiMo **found nothing**. It read both regions and flagged neither — and the *Regions read* tab shows
why: its answer for the payload-bearing region came back unparseable, one of the 155-in-1,100
formatting failures the evaluation counted as *not injected*.

The family-naming model, reading the same file through four other signals, names it
**`ransomware_simulation` at 100% confidence**, and quotes **P@1 0.994** because this file carries a
known signature. The label is correct: `Example_5.pdf` is `ransomware_simulation_AtomicRedTeam_0751`.

So the two models disagree, in public, in the report — and on this file the language model is the one
that is wrong. That is the argument for stage 4 in a single screenshot.

## The five tabs

- **Report** — the verdict for this batch, the family with its confidence and the accuracy figure for
  this file's regime, and the containment advice for that family. It always states how many regions
  in other batches are unread, and how many answers could not be parsed (those count as *not
  injected*, exactly as the evaluation scored them).
- **Regions read** — one row per region: where in the skeleton, `marker` or `sweep`, which signatures
  the regex saw, MiMo's verdict, and the substring quoted as evidence.
- **Family shortlist** — the model's ranked top three of thirteen, with confidences.
- **Nearest corpus files** — the five most similar corpus documents to the flagged region.
  Precision@5 is 35.6%, so read it as *resemblance*, never identification.
- **What MiMo actually said** — the untouched generation per region. This is where to look when a
  verdict seems wrong.

![The family shortlist tab](docs/screenshots/07_family_shortlist.png)

*The shortlist, ranked with confidences. On this file the model is unusually decisive; on a document
carrying no known signature the second and third rows carry real weight, which is why three are shown
rather than one.*

![The regions read tab](docs/screenshots/09_regions_read.png)

*Region 1 is the marker region — signatures `javascript_injection, ransomware_simulation` — and
MiMo's verdict is `clean (unreadable answer)`. Region 2 is a sweep region, genuinely clean. This tab
is what turns "the model said nothing" into "the model failed to format an answer", which are very
different failures.*

![The nearest corpus files tab](docs/screenshots/08_nearest_corpus.png)

*All five neighbours are `ransomware_simulation`, the closest at 0.974 — and that closest file,
`ransomware_simulation_AtomicRedTeam_0751.pdf`, **is** the corpus original of `Example_5.pdf`. This is
the self-match described above, visible in the interface: the shipped examples are corpus documents,
so they find themselves. Upload a PDF of your own and this table gets harder.*

**The two models are allowed to disagree in public.** MiMo can flag a region the family model puts in
the `none` class, and the family model routinely overrules MiMo's guess. The report prints the
disagreement rather than resolving it silently, because two models disagreeing is a reason to look at
the file yourself.

### The nearest-files tab is a recommender, and the score is cosine similarity

Content-based rather than collaborative — there are no users to learn from, only documents. The
flagged region is embedded with the winning model (`nomic-ai/nomic-embed-text-v1.5`, the
`search_document: ` prefix, 768 dimensions) into the same space as the 1,100 corpus files, and the
five most similar are returned.

Both index and query are **unit-normalised**, so the dot product *is* the cosine similarity and the
whole lookup is one 1,100 × 768 matrix–vector product — no approximate index is needed at this size
and the ranking is exact. `check_provenance()` asserts the model, prefix, dimension, normalisation
and source column against the retrieval stage's own results file before any lookup runs, because a
query embedded differently lands in a different space and returns confident nonsense with no error
anywhere.

The same query vector feeds the family model's neighbour-vote block, which is why one checkbox
enables both.

## Where the plan panel's numbers come from

This block appears under the button as soon as a file is loaded, and it is written **before any model
runs** — pure text extraction and regex, recomputed on every upload, slider move and checkbox change.

![The plan panel](docs/screenshots/04_plan_panel.png)

**Line 1 — the extraction.**

- **`12,729 bytes` on disk** — the length of the uploaded file, unmodified.
- **`5,255-character skeleton`** — the result of `build_skeleton()`. Every `stream … endstream` body
  is inflated with `zlib` where it is FlateDecode, capped at 4,096 characters, and kept if more than
  60% of its bytes are printable ASCII. Control and binary residue becomes spaces; runs of four or
  more spaces collapse to three.
- **`4 binary stream(s) dropped`** — streams that failed the 60%-printable test, i.e. genuine binary
  such as an image. Each is replaced by the literal `<<BINARY n bytes>>` rather than deleted, so its
  position is preserved.
- If the file is very large the line also says **truncated**: past a 120,000-character budget the
  skeleton keeps 45% from the head and the rest from the tail, because the generator inserts payloads
  at both ends and a plain head truncation would lose most of them.

**Line 2 — the triage.** `ANY_MARKER_RE`, a single alternation of all twelve families' structural
patterns plus the framework and test-payload strings, is run over the whole skeleton.

- **`1 region(s) carry a marker`** — every match becomes a ±1,500-character window centred on the
  hit, and overlapping windows are merged, so this is regions after merging rather than raw matches.
- **`the remaining 1 cover the rest of the document`** — the sweep. The skeleton is tiled into
  3,000-character windows, and a tile is dropped if a signature region already covers half of it.
- **`2 in total, cut into 1 batch(es) of at most 8`** — the two lists are concatenated, signature
  regions first in density order, then sliced into batches.
- **`Structural signatures in the raw file: …`** — which families' regexes matched, run against the
  **raw bytes**, not the skeleton. Two names appear for several files because the generator gives
  some families a JavaScript launcher.

Two things this panel deliberately does not do. It does not print `javascript_injection` on its own
as though that were a finding — plenty of harmless PDFs contain a `/JS (` token, and the word
*signature* is doing real work. And the ordering decides **reading order only**: batch 1 is the most
signature-dense, not the guilty one.

---

# Part 3 — The models and the numbers

On the 1,100-document corpus, MiMo-7B-RL scored:

| | |
|---|---|
| F1 | **0.945** |
| precision | 0.988 |
| recall | 0.906 |
| names the family correctly | 43.3% of files it caught |
| false alarms | 10 of 200 clean files (5%) |
| unparsable answers | 155 of 1,100 |

**A detector that calls every file malicious scores F1 0.900 on this corpus**, because 82% of it is
injected. Read 0.945 against 0.900, not against zero.

## Why MiMo, when Gemma scored higher

The evaluation's actual winner is **Gemma-2-9B at F1 0.969**; this app runs **MiMo-7B at 0.945**.
That is a hosting decision, not a disagreement with the evaluation:

- **Gemma is gated.** It needs an account with Google's licence accepted plus a read token. On a
  public Space that means a new visitor meets a 403, or the app asks strangers to paste a token.
  MiMo downloads for anyone.
- **Gemma is 2.6× slower** — 10.95 s per window against 4.18 s on the same T4. On ZeroGPU a grant is
  capped at 300 s and the model load must fit inside it too.
- **Free ZeroGPU is about five minutes per day.** At Gemma's rate that is a couple of batches.

What it costs is **0.024 F1**. Family-naming accuracy also differs — Gemma 63%, MiMo 43% — but that
no longer decides anything: the family is named by the model in Part 1, which reaches 0.993 without
MiMo's guess at all. Choosing the faster, ungated model is a cheaper decision than it was when the
family label came straight from the language model.

## One runtime

**ZeroGPU, running the evaluation's own configuration.** The BF16 checkpoint of
`XiaomiMiMo/MiMo-7B-RL` quantised to 4-bit NF4 by `bitsandbytes`, greedy, 200 new tokens, batched at
8. Nothing about the arithmetic differs from the run that produced F1 0.945. The whole scan happens
inside a single `@spaces.GPU` call, because ZeroGPU grants and reclaims the device around each one
and paying the model-load cost per region would dominate everything else.

One grant is kept short — 110 s requested, 165 s reserved — and a batch tops out at 8 regions,
because the scheduler reserves the full requested duration up front.

**There is no `gpu`/`cpu` control.** A CPU path through `llama.cpp` was attempted so the Space could
keep working after a visitor's daily quota ran out, and it cannot be installed here at all: the
prebuilt `llama-cpp-python` wheels are tagged `linux_x86_64` but linked against **musl** while a
Space runs on glibc, and PyPI ships no binary wheel, so the sdist has to compile — which exceeded the
build limit with `Job timeout`. A control whose second option always fails is worse than one option.

### Quotas, and what "MiMo could not run" means

![The ZeroGPU quota message](docs/screenshots/10_quota.png)

*The one screenshot here not from the session above, and it could not be: reproducing it requires an
exhausted allowance. It was captured from a signed-out visit, which is also why it is styled
differently — this is precisely what an anonymous visitor sees. Note the arithmetic: 165 s requested
against 164 s remaining, refused by one second.*

A free visitor gets roughly five minutes of ZeroGPU per day. The quota is counted per visitor, not
per Space: this message means your allowance, not a broken app. Signing in to Hugging Face raises it,
which is the practical fix.

## Triage and the sweep

**It triages instead of scanning everything.** A grant is capped at 300 seconds while a real PDF has
dozens of 3,000-character windows, so reading all of them is not on offer. The same signature
alternation that located the payload in the corpus is run over the whole skeleton, every hit becomes
a candidate window of identical shape, overlapping ones are merged, and the most signature-dense go
first. A file with no signature anywhere yields exactly one candidate — the head of the document —
byte-identical to what the corpus builder produced for a clean file.

**The sweep regions are honestly worse, and the app says so.** The evaluation only ever showed MiMo
signature-centred windows or the head of a document. Handed an arbitrary mid-file content stream — a
page of font-positioning operators — MiMo frequently does not answer at all: it carries on copying
the input after the prefill and the answer parses as unrecoverable, which scores as *not injected*.
Those regions buy coverage of text that would otherwise never be looked at; a *clean* verdict on one
is close to no evidence. The regions table labels every row, the report counts sweep parse failures
separately, and the sweep can be switched off.

**Nothing is downloaded until it is needed.** The page comes up first; MiMo's checkpoint, the 550 MB
embedding model and the 3 MB index arrive on the first scan and are cached. The family-naming model
is the exception and not one worth optimising — at 915 KB it ships in the repo.

---

# Part 4 — Provenance, and the files

## The three repos this is built on

| Repo | What this Space takes from it |
|---|---|
| [Generated_Injected_PDFs_HARMLESS](https://huggingface.co/datasets/Cyber-security-final-project/Generated_Injected_PDFs_HARMLESS) | The 12 injection families and their structural signatures — the definition of what an attack looks like |
| [HARMLESS_Synthetic_Injected_PDFs_EDA](https://huggingface.co/datasets/Cyber-security-final-project/HARMLESS_Synthetic_Injected_PDFs_EDA) | `build_skeleton`, `mask_leaks`, `payload_window` and the 17 numeric features |
| [Evaluation_of_OpenSource_Models…](https://huggingface.co/datasets/Cyber-security-final-project/Evaluation_of_OpenSource_Models_for_PDF_Injection_Recognition) | The embedding index and winning configuration; the prompt, prefill and parser |

The prompt, the prefill, the parser, the whole text-extraction path and the seventeen document-shape
features are quoted **verbatim** from those notebooks. That is the correctness argument for quoting
the published scores here at all: change how the text is extracted or how the question is asked, and
those numbers stop describing this program.

`test_fidelity.py` enforces it. It pulls real PDFs from the generation repo, runs them through
`corpus_text.py`, and compares the result **character by character** against the published parquet —
then recomputes all 17 numeric features and checks them against the published columns, and asserts
the assembled feature row is the width the fitted model expects.

```
python test_fidelity.py 8
```

## The examples

Thirteen documents ship with the Space — one per injection family plus a clean control. On disk they
are `Example_0` … `Example_12`, so the uploader shows a neutral filename. The buttons are labelled
because a demo you can drive deliberately is more useful than a guessing game, and it costs nothing
on the model side since a filename never reaches a prompt.

The numbering is shuffled with a fixed seed, and every file sits in a **12–15 KB band** so file size
does not reveal the clean control. Each produces 2–8 regions, so every one fits in a single batch.
`examples/Example_Key.txt` records which is which.

**They flatter the family model**, and the reason is worth knowing: every example is a corpus file,
and the corpus is the neighbour index, so each finds *itself* among its neighbours at similarity ≈
1.0 — the self-match that cross-validation masks out and serving cannot. Upload a PDF of your own to
see the model work without that help.

## The files

| File | What it does |
|---|---|
| `app.py` | The Gradio interface and the document-level report. No detection logic. |
| `corpus_text.py` | PDF bytes → skeleton → candidate windows. Verbatim from the EDA notebook above the `Triage` heading. |
| `mimo.py` | The prompt, prefill and parser, verbatim from the evaluation, on the 4-bit GPU runtime. |
| `neighbours.py` | The embedding index and nearest-neighbour lookup, with a provenance assertion. |
| `doc_features.py` | The 17 document-shape measurements, verbatim from the EDA notebook's `extract_one`. |
| `family_model.py` | Feature assembly and the family-naming model's ranked prediction. |
| `family_naming_model.pkl` | The fitted classifier and the feature schema it expects, in one file. |
| `test_fidelity.py` | Checks extraction *and* features against the published corpus. |
| `App_Creation.ipynb` | The build log: fits the model in Part 2, documents the app in Parts 3–9. |
| `figures/` | The charts in Part 1 of this README. |

## Secrets

None. Every model used here is ungated.