README.md

---
library_name: transformers
pipeline_tag: text-classification
tags:
  - cybersecurity
  - blue-team
  - log-analysis
  - text-classification
  - custom-code
  - pytorch
language:
  - en
license: mit
---

# Tiny Blue Log Classifier

A very small CPU-friendly log classifier for blue-team experiments and defensive security workflows.

This model classifies a log line into one of two labels:

- `BENIGN`
- `SUSPICIOUS`

It uses a custom Hugging Face Transformers architecture and tokenizer stored directly in this repository.

## Intended use

This project is intended for:

- blue-team experimentation
- log triage prototypes
- learning how custom Hugging Face models work
- low-resource CPU deployments


The included checkpoint was trained on a small synthetic demonstration dataset. Treat its predictions as experimental triage signals, not authoritative security verdicts.

## Model size

| Property | Value |
|---|---:|
| Parameters | 16,418 |
| Vocabulary buckets | 1,024 |
| Embedding size | 16 |
| Output labels | 2 |
| Maximum input length | 96 tokens |
| GPU required | No |
| Target deployment | CPU |
| Suggested minimum VM | 2 CPU cores, 2 GB RAM |

Architecture:

```text
log text
   ↓
custom normalization
   ↓
hashed tokenizer
   ↓
Embedding(1024, 16)
   ↓
mean pooling
   ↓
Linear(16, 2)
   ↓
BENIGN / SUSPICIOUS
```

## Installation

Create a Python virtual environment:

```bash
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
```

For a CPU-only Linux machine:

```bash
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install transformers safetensors
```

## Basic usage

```python
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

repo = "mozarilla/tiny-blue-log-classifier"

torch.set_num_threads(2)

tokenizer = AutoTokenizer.from_pretrained(
    repo,
    trust_remote_code=True,
)

model = AutoModelForSequenceClassification.from_pretrained(
    repo,
    trust_remote_code=True,
)

model.eval()

log = (
    "EventID=4625 Failed logon "
    "user=administrator "
    "source_ip=203.0.113.44 "
    "count=17"
)

inputs = tokenizer(
    log,
    return_tensors="pt",
    truncation=True,
    max_length=96,
)

with torch.inference_mode():
    logits = model(**inputs).logits
    probabilities = torch.softmax(logits, dim=-1)[0]

prediction_id = int(probabilities.argmax().item())
label = model.config.id2label[prediction_id]
confidence = float(probabilities[prediction_id])

print({
    "label": label,
    "confidence": confidence,
})
```

Example output:

```text
{
    'label': 'SUSPICIOUS',
    'confidence': 0.93
}
```

The exact score may change between model revisions.

## Quick test

```python
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

repo = "mozarilla/tiny-blue-log-classifier"

tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForSequenceClassification.from_pretrained(
    repo,
    trust_remote_code=True,
)

text = "Windows Defender scan completed host=WS-014 threats=0 status=clean"

inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=96)

with torch.inference_mode():
    probs = torch.softmax(model(**inputs).logits, dim=-1)[0]

idx = int(probs.argmax())

print(model.config.id2label[idx], float(probs[idx]))
```

## Example logs

Successful login:

```text
EventID=4624 Successful logon user=alice source_ip=10.0.0.15 logon_type=2
```

Repeated failed login:

```text
EventID=4625 Failed logon user=administrator source_ip=203.0.113.44 count=17
```

Audit log cleared:

```text
EventID=1102 The audit log was cleared subject_user=svc-backup host=DC-01
```

Clean Defender scan:

```text
Windows Defender scan completed host=WS-014 threats=0 status=clean
```

## Classify a file

This repository includes `classify_file.py`.

For a text file containing one log event per line:

```bash
python classify_file.py \
  mozarilla/tiny-blue-log-classifier \
  sample.log \
  --output classified.jsonl
```

Example output:

```json
{"line": 1, "label": "BENIGN", "suspicious_probability": 0.023, "text": "..."}
{"line": 2, "label": "SUSPICIOUS", "suspicious_probability": 0.932, "text": "..."}
```

The file classifier processes logs one line at a time to keep memory usage low.

### Model implementation

`modeling_tiny_log.py` defines:

```python
TinyLogForSequenceClassification
```

The model performs:

```text
token IDs
   ↓
embedding lookup
   ↓
masked mean pooling
   ↓
linear classifier
```

### Tokenizer implementation

`tokenization_tiny_log.py` defines:

```python
TinyLogTokenizer
```

It calls the normalization and hashing functions in `tinylog_core.py`.

### AutoClass mapping

`config.json` maps the standard Transformers API to the custom model:

```json
{
  "auto_map": {
    "AutoConfig": "configuration_tiny_log.TinyLogConfig",
    "AutoModelForSequenceClassification": "modeling_tiny_log.TinyLogForSequenceClassification"
  }
}
```

`tokenizer_config.json` maps `AutoTokenizer` to the custom tokenizer:

```json
{
  "auto_map": {
    "AutoTokenizer": [
      "tokenization_tiny_log.TinyLogTokenizer",
      null
    ]
  }
```

## How logs are processed

The tokenizer performs lightweight normalization.

Examples:

```text
192.168.1.50  ->  <ip>
15:46:23      ->  <time>
long hex      ->  <hex>
UUID          ->  <uuid>
```

Tokens are deterministically hashed into a fixed vocabulary of 1,024 buckets. This keeps the tokenizer and model extremely small.

## Labels

### BENIGN

The log appears closer to benign patterns represented in the training data.

### SUSPICIOUS

The log appears closer to suspicious patterns represented in the training data.

`SUSPICIOUS` does **not** mean that an event has been proven malicious.

A security analyst should combine the result with surrounding events, process ancestry, user identity, host role, network context, threat intelligence, detection rules, and endpoint telemetry.

## Limitations

1. The demonstration training data is synthetic.
2. The model has only 16,418 parameters.
3. It does not understand long event sequences or relationships between multiple logs.
4. Hash collisions can occur because tokens are mapped into only 1,024 buckets.
5. It is not a replacement for signature-based or behavioral detection systems.
6. A high `SUSPICIOUS` score is not proof of malicious activity.
7. A `BENIGN` prediction is not proof that an event is safe.
8. Logs from formats not represented during training may produce unreliable predictions.

For meaningful deployment, retrain the classifier on reviewed logs representative of your own environment.

## Recommended production pattern

```text
logs
  ↓
normalization
  ↓
existing detection rules
  ↓
Tiny Blue Log Classifier
  ↓
risk score / enrichment
  ↓
SIEM or analyst queue
```

Do not automatically block users, isolate hosts, delete files, or take other destructive actions based only on this model's output.

## Repository files

```text
README.md
SECURITY.md
LICENSE
config.json
tokenizer_config.json
vocab_config.json
model.safetensors
configuration_tiny_log.py
modeling_tiny_log.py
tokenization_tiny_log.py
tinylog_core.py
infer_hf.py
classify_file.py
requirements-runtime.txt
```

## License

MIT