---
license: mit
language:
  - en
  - multilingual
library_name: transformers
tags:
  - password-generator
  - password-cracking
  - cybersecurity
  - red-team
  - password-analysis
  - llm
  - penetration-testing
  - passguess
  - pml-6l
pipeline_tag: text-generation

---

# PML-6L -- Password Modeling Language

This repository contains a password modeling language trained from scratch on 13 million real-world passwords. Built on a LLaMA-style architecture (RoPE + SwiGLU + RMSNorm) with only 11M parameters, it learns the statistical structure of human-created passwords and can generate realistic candidates for authorized security testing.

## Architecture

| Parameter | Value |
|-----------|-------|
| Layers | 6 |
| Hidden dimension | 384 |
| Attention heads | 6 |
| Head dimension | 64 |
| **Total parameters** | **11,005,824** |
| Vocabulary size | 500 (BPE) |
| Max sequence length | 24 |
| Position encoding | RoPE (Rotary Position Embeddings) |
| Activation | SwiGLU (Gated Linear Unit) |
| Normalization | RMSNorm |
| Attention | FlashAttention (via PyTorch SDPA) |
| Training precision | FP16 (Automatic Mixed Precision) |

## Training Data

| Source | Train | Validation |
|--------|-------|------------|
| [rockyou.txt](https://weakpass.com/wordlists/rockyou.txt) (90/10 split) | ~12.9M | ~1.4M |
| [sqlmap.txt](https://weakpass.com/wordlists/sqlmap.txt) (10/90 split) | ~120K | ~1.1M |
| **Total** | **~13.0M** | **~2.2M** |

Train/validation split is **disjoint** -- no password appears in both sets. Passwords are filtered to 4-24 characters, printable ASCII only.

### Evaluation

The model is evaluated using **hashcat v7.1.2** (RTX 4060, 27.5 GH/s MD5). Target: **hashkiller.dic** (23.7M passwords, MD5 hashed). We generate 5,000 passwords per shot across 5 temperatures, 5 presets, and 4 epochs (100 shots total). Three rule sets show how rules amplify cracking power:

| Mode | Rule Set | Rules | What It Shows |
|------|----------|-------|---------------|
| **RAW** | None | 0 | Model's direct password quality |
| **LIGHT** | `top_500.rule` | 500 | Cracking with common mutations |
| **FULL** | `OneRuleToRuleThemAll.rule` | 52,016 | Maximum possible coverage |

For RAW and LIGHT, per-shot stats come from hashcat `--stdout` + potfile matching. FULL per-shot is proportionally scaled from LIGHT.

### Overall Results

| Mode | Total Unique PWs Tested | Unique Hashes Cracked | Avg per Shot |
|------|------------------------|----------------------|-------------|
| RAW | 217,789 | 31,005 | 2,220 |
| LIGHT | 217,789 | 984,164 (potfile) | 4,955 |
| FULL | 217,789 | 4,496,654 (potfile) | 22,646 |

### Detailed RAW Results (Best Preset: L8_MIXED)

| Epoch | T=0.2 | T=0.4 | T=0.6 | T=0.8 | T=1.0 | Avg | Best |
|-------|-------|-------|-------|-------|-------|-----|------|
| 1 | 2,847 | 3,668 | 3,475 | 3,091 | 2,753 | 3,167 | 3,668 |
| 2 | 3,447 | 3,911 | 3,608 | 3,185 | 2,827 | 3,396 | 3,911 |
| 3 | **4,189** | 4,163 | 3,765 | 3,321 | 2,970 | **3,682** | 4,189 |
| 4 | **4,293** | 4,187 | 3,654 | 3,149 | 2,775 | 3,612 | **4,293** |

### Winners by Preset

#### RAW (Direct Match)

| Preset | Best Avg Epoch | Avg Cracked | Best Shot | Details |
|--------|---------------|-------------|-----------|---------|
| L8_MIXED | Epoch 3 | 3,682 | **4,293** (Ep4 T=0.2) | Mixed case + specials |
| L6_ALPHA | Epoch 3 | 2,623 | 3,834 (Ep1 T=0.2) | Alphabetic only |
| L12_ALPHANUM | Epoch 4 | 4,777 | 4,988 (Ep4 T=0.2) | 12-char alphanumeric |
| L10_ALL | Epoch 4 | 286 | 671 (Ep4 T=0.2) | 10-char all types |
| plain | Epoch 4 | 311 | 609 (Ep4 T=0.2) | No conditional tag |

#### LIGHT (500 Rules) / FULL (52K Rules)

With rule application, most presets saturate near 100% coverage per shot:

| Preset | LIGHT Avg | FULL Avg | LIGHT Best Shot | FULL Best Shot |
|--------|-----------|----------|----------------|----------------|
| L8_MIXED | 5,000 | 22,845 | 5,000 (all) | 22,845 (all) |
| L6_ALPHA | 5,000 | 22,845 | 5,000 (all) | 22,845 (all) |
| L12_ALPHANUM | 5,000 | 22,845 | 5,000 (all) | 22,845 (all) |
| L10_ALL | 4,916 | 22,462 | 5,000 (Ep4 T=0.2) | 22,845 |
| plain | 4,954 | 22,633 | 5,000 (Ep2 T=0.2) | 22,845 |

### Temperature Sensitivity

Lower temperatures produce more common (higher-probability) passwords, resulting in more direct matches in the target dictionary. Higher temperatures increase diversity at the cost of raw hit rate.

| Temp | RAW Avg | LIGHT Avg | FULL Avg |
|------|---------|-----------|----------|
| 0.2 | **2,614** | **4,997** | **22,831** |
| 0.4 | 2,508 | 4,998 | 22,835 |
| 0.6 | 2,208 | 4,991 | 22,803 |
| 0.8 | 1,972 | 4,947 | 22,605 |
| 1.0 | 1,801 | 4,840 | 22,114 |

**Recommended configuration:** Temperature **0.6** for best balance of hit rate and diversity (0% duplicate rate). Temperature **0.2** for maximum raw crack count.

## Training Configuration

| Hyperparameter | Value |
|----------------|-------|
| Optimizer | AdamW (β₁=0.9, β₂=0.95) |
| Learning rate | 3e-4 → 3e-5 (cosine decay) |
| Warmup steps | 1,000 |
| Weight decay | 0.1 |
| Gradient clipping | 1.0 |
| Batch size | 256 |
| Gradient accumulation | 8 steps |
| Effective batch size | 2,048 |
| Epochs | 4 (early stopped) |

## Loss Progression

| Epoch | Train Loss | Val Loss | Δ Val | Perplexity (Val) |
|-------|-----------|----------|-------|-----------------|
| 1 | 0.9348 | 1.0643 | -- | 2.90 |
| 2 | 0.8556 | 1.0605 | -0.0038 | 2.89 |
| 3 | 0.8522 | 1.0573 | -0.0032 | 2.88 |
| 4 | 0.8501 | 1.0600 | +0.0027 | 2.89 |

**Best checkpoint: Epoch 3** (lowest validation loss). Overfitting begins at epoch 4, indicated by the first val loss increase.

## Generation Samples

### Temperature Sweep (`[LEN:12][CHARS:ALL]` tag)

| Temperature | Samples |
|-------------|---------|
| 0.2 | `Christian_`, `Johnny_19`, `GABYGirl`, `IloveJoe_` |
| 0.4 | `Beautiful`, `Angel_19`, `Chris@1980`, `Bert_Ro` |
| 0.6 | `Chris_1985`, `Angel_200`, `KYLIE@`, `HeartEy10` |
| 0.8 | `Charlyne1!`, `Abcd_12`, `Love@@s`, `M@r1m@` |
| 1.0 | `#1POKE`, `**Mir3l`, `LaGefg`, `ACUSTA` |

### Conditional Generation

The model supports tag-based conditional generation for targeted password candidates:

```python
# Plain generation (no tag)
>>> model.generate(tokenizer, temperature=0.6)
"iloveyou123"

# 8-character alphanumeric password
>>> prefix = tokenizer.encode("[LEN:8][CHARS:ALPHANUM]:").ids
>>> model.generate(tokenizer, temperature=0.6, prefix_ids=prefix)
"Johnny19"

# 12-character mixed password (letters + numbers + specials)
>>> prefix = tokenizer.encode("[LEN:12][CHARS:ALL]:").ids
>>> model.generate(tokenizer, temperature=0.8, prefix_ids=prefix)
"Angel_2000!"

# 6-character alphabetic (lowercase only)
>>> prefix = tokenizer.encode("[LEN:6][CHARS:ALPHA]:").ids
>>> model.generate(tokenizer, temperature=0.4, prefix_ids=prefix)
"dabook"
```

### Expected Output by Temperature

| Temperature | Diversity | Duplicate Rate | Recommended Use |
|-------------|-----------|---------------|-----------------|
| 0.2 | Low | ~30-35% | Most common passwords |
| 0.4 | Medium | ~4-5% | Balanced (recommended) |
| 0.6 | High | ~0% | General purpose (recommended) |
| 0.8 | Very High | ~0% | Diverse candidates |
| 1.0 | Maximum | ~0% | Creative/rare patterns |

## Usage

### Installation

```bash
pip install torch tokenizers
```

### Load Model

```python
import torch
from tokenizers import Tokenizer
from model_v2 import LLaMAModel

device = "cuda" if torch.cuda.is_available() else "cpu"

tokenizer = Tokenizer.from_file("tokenizer.json")
model = LLaMAModel.from_pretrained("K0D3IN/PML-6L").to(device)
model.eval()
```

### Generate Passwords

```python
def generate(model, tokenizer, temperature=0.6, tag=None, n=10):
    prefix_ids = tokenizer.encode(tag).ids if tag else None
    pws = set()
    while len(pws) < n:
        pw = model.generate(
            tokenizer,
            temperature=temperature,
            top_k=50,
            max_len=24,
            min_len=4,
            prefix_ids=prefix_ids,
            device=device,
        )
        if pw:
            pws.add(pw)
    return list(pws)

# Generate 50 passwords for security testing
passwords = generate(model, tokenizer, temperature=0.6, n=50)
```

## Training Details

This model was trained on an **NVIDIA GeForce RTX 4060 Laptop GPU (8GB VRAM)** using PyTorch 2.11 with CUDA 13.2.

| Metric | Value |
|--------|-------|
| Training time (4 epochs) | ~2 hours 40 minutes |
| Tokens/second | ~123,000 |
| GPU memory usage | ~190 MB |
| Checkpoint size | 126 MB |

## Architecture Diagram (Simplified)

```
Input IDs → Embedding (vocab=500 → dim=384)
          → RoPE Position Encoding
          → 6× Block:
              ├── RMSNorm
              ├── Causal Self-Attention (6 heads, FlashAttention)
              ├── RMSNorm
              └── SwiGLU MLP (384 → 1024 → 384)
          → RMSNorm
          → LM Head (384 → vocab=500)
          → Logits
```

## Intended Use

| ✅ Appropriate | ❌ Inappropriate |
|---------------|-----------------|
| Authorized penetration testing | Credential stuffing against live systems |
| Red team engagements with written authorization | Account takeover |
| Password policy research | Illegal access to systems |
| Security awareness training | Privacy violations |
| Academic password research | Harassment or doxxing |
| Internal security audits | Any use without explicit authorization |

## Ethical Notice

**This model generates password candidates that may match real user credentials.**

Passwords generated by this model are **statistical patterns** derived from public breach data (rockyou.txt and sqlmap.txt). They do not contain actual user credentials from the training set. However, due to the nature of password generation, some outputs may coincidentally match real passwords currently in use.

### Users are solely responsible for:
- Obtaining proper authorization before any security testing
- Complying with all applicable laws (CFAA, GDPR, KVKK, etc.)
- Ensuring testing is conducted within authorized scope only
- Secure handling and disposal of generated password lists

### This model should NOT be used for:
- Any illegal activity
- Attacking systems without explicit written permission
- Mass credential stuffing or account takeover attempts
- Harassing individuals or organizations



## Files

| File | Size | Description |
|------|------|-------------|
| `model.safetensors` | ~45 MB | Model weights |
| `config.json` | ~1 KB | HF model configuration |
| `tokenizer.json` | ~50 KB | BPE tokenizer (vocab=500) |
| `model_v2.py` | ~8 KB | Model definition code |
| `generation_samples.txt` | ~10 KB | Example outputs at all temperatures |

## Citation

```bibtex
@software{PML-6L,
  author = {K0D3IN},
  title = {PML-6L: Password Pattern Generator},
  year = {2026},
  url = {https://huggingface.co/K0D3IN/PML-6L}
}
```

## License

This project is licensed under the **MIT License** -- see the [LICENSE](LICENSE) file for details.

## Support Open Source AI Research

Every donation helps maintain and improve this project:

**Monero (XMR):**
```
83iqXtvVu28ZiL9bsATMerSgbFFiD1J1jc96CcxJLEnAW3KBmBKedWnUAeLvLvEA9aBiUBpHQJs1iNHYtkTLZbNUEymobSS
```

**Bitcoin (BTC):**
```
bc1qmnlvpukcgl0hsr7nje0x8555mhtxjt80wtmlxm
```