PML-6L -- Password Modeling Language

This repository contains a password modeling language trained from scratch on 13 million real-world passwords. Built on a LLaMA-style architecture (RoPE + SwiGLU + RMSNorm) with only 11M parameters, it learns the statistical structure of human-created passwords and can generate realistic candidates for authorized security testing.

Architecture

Parameter	Value
Layers	6
Hidden dimension	384
Attention heads	6
Head dimension	64
Total parameters	11,005,824
Vocabulary size	500 (BPE)
Max sequence length	24
Position encoding	RoPE (Rotary Position Embeddings)
Activation	SwiGLU (Gated Linear Unit)
Normalization	RMSNorm
Attention	FlashAttention (via PyTorch SDPA)
Training precision	FP16 (Automatic Mixed Precision)

Training Data

Source	Train	Validation
rockyou.txt (90/10 split)	~12.9M	~1.4M
sqlmap.txt (10/90 split)	~120K	~1.1M
Total	~13.0M	~2.2M

Train/validation split is disjoint -- no password appears in both sets. Passwords are filtered to 4-24 characters, printable ASCII only.

Evaluation

The model is evaluated using hashcat v7.1.2 (RTX 4060, 27.5 GH/s MD5). Target: hashkiller.dic (23.7M passwords, MD5 hashed). We generate 5,000 passwords per shot across 5 temperatures, 5 presets, and 4 epochs (100 shots total). Three rule sets show how rules amplify cracking power:

Mode	Rule Set	Rules	What It Shows
RAW	None	0	Model's direct password quality
LIGHT	top_500.rule	500	Cracking with common mutations
FULL	OneRuleToRuleThemAll.rule	52,016	Maximum possible coverage

For RAW and LIGHT, per-shot stats come from hashcat --stdout + potfile matching. FULL per-shot is proportionally scaled from LIGHT.

Overall Results

Mode	Total Unique PWs Tested	Unique Hashes Cracked	Avg per Shot
RAW	217,789	31,005	2,220
LIGHT	217,789	984,164 (potfile)	4,955
FULL	217,789	4,496,654 (potfile)	22,646

Detailed RAW Results (Best Preset: L8_MIXED)

Epoch	T=0.2	T=0.4	T=0.6	T=0.8	T=1.0	Avg	Best
1	2,847	3,668	3,475	3,091	2,753	3,167	3,668
2	3,447	3,911	3,608	3,185	2,827	3,396	3,911
3	4,189	4,163	3,765	3,321	2,970	3,682	4,189
4	4,293	4,187	3,654	3,149	2,775	3,612	4,293

Winners by Preset

RAW (Direct Match)

Preset	Best Avg Epoch	Avg Cracked	Best Shot	Details
L8_MIXED	Epoch 3	3,682	4,293 (Ep4 T=0.2)	Mixed case + specials
L6_ALPHA	Epoch 3	2,623	3,834 (Ep1 T=0.2)	Alphabetic only
L12_ALPHANUM	Epoch 4	4,777	4,988 (Ep4 T=0.2)	12-char alphanumeric
L10_ALL	Epoch 4	286	671 (Ep4 T=0.2)	10-char all types
plain	Epoch 4	311	609 (Ep4 T=0.2)	No conditional tag

LIGHT (500 Rules) / FULL (52K Rules)

With rule application, most presets saturate near 100% coverage per shot:

Preset	LIGHT Avg	FULL Avg	LIGHT Best Shot	FULL Best Shot
L8_MIXED	5,000	22,845	5,000 (all)	22,845 (all)
L6_ALPHA	5,000	22,845	5,000 (all)	22,845 (all)
L12_ALPHANUM	5,000	22,845	5,000 (all)	22,845 (all)
L10_ALL	4,916	22,462	5,000 (Ep4 T=0.2)	22,845
plain	4,954	22,633	5,000 (Ep2 T=0.2)	22,845

Temperature Sensitivity

Lower temperatures produce more common (higher-probability) passwords, resulting in more direct matches in the target dictionary. Higher temperatures increase diversity at the cost of raw hit rate.

Temp	RAW Avg	LIGHT Avg	FULL Avg
0.2	2,614	4,997	22,831
0.4	2,508	4,998	22,835
0.6	2,208	4,991	22,803
0.8	1,972	4,947	22,605
1.0	1,801	4,840	22,114

Recommended configuration: Temperature 0.6 for best balance of hit rate and diversity (0% duplicate rate). Temperature 0.2 for maximum raw crack count.

Training Configuration

Hyperparameter	Value
Optimizer	AdamW (β₁=0.9, β₂=0.95)
Learning rate	3e-4 → 3e-5 (cosine decay)
Warmup steps	1,000
Weight decay	0.1
Gradient clipping	1.0
Batch size	256
Gradient accumulation	8 steps
Effective batch size	2,048
Epochs	4 (early stopped)

Loss Progression

Epoch	Train Loss	Val Loss	Δ Val	Perplexity (Val)
1	0.9348	1.0643	--	2.90
2	0.8556	1.0605	-0.0038	2.89
3	0.8522	1.0573	-0.0032	2.88
4	0.8501	1.0600	+0.0027	2.89

Best checkpoint: Epoch 3 (lowest validation loss). Overfitting begins at epoch 4, indicated by the first val loss increase.

Generation Samples

Temperature Sweep ([LEN:12][CHARS:ALL] tag)

Temperature	Samples
0.2	Christian_, Johnny_19, GABYGirl, IloveJoe_
0.4	Beautiful, Angel_19, Chris@1980, Bert_Ro
0.6	Chris_1985, Angel_200, KYLIE@, HeartEy10
0.8	Charlyne1!, Abcd_12, Love@@s, M@r1m@
1.0	#1POKE, **Mir3l, LaGefg, ACUSTA

Conditional Generation

The model supports tag-based conditional generation for targeted password candidates:

# Plain generation (no tag)
>>> model.generate(tokenizer, temperature=0.6)
"iloveyou123"

# 8-character alphanumeric password
>>> prefix = tokenizer.encode("[LEN:8][CHARS:ALPHANUM]:").ids
>>> model.generate(tokenizer, temperature=0.6, prefix_ids=prefix)
"Johnny19"

# 12-character mixed password (letters + numbers + specials)
>>> prefix = tokenizer.encode("[LEN:12][CHARS:ALL]:").ids
>>> model.generate(tokenizer, temperature=0.8, prefix_ids=prefix)
"Angel_2000!"

# 6-character alphabetic (lowercase only)
>>> prefix = tokenizer.encode("[LEN:6][CHARS:ALPHA]:").ids
>>> model.generate(tokenizer, temperature=0.4, prefix_ids=prefix)
"dabook"

Expected Output by Temperature

Temperature	Diversity	Duplicate Rate	Recommended Use
0.2	Low	~30-35%	Most common passwords
0.4	Medium	~4-5%	Balanced (recommended)
0.6	High	~0%	General purpose (recommended)
0.8	Very High	~0%	Diverse candidates
1.0	Maximum	~0%	Creative/rare patterns

Usage

Installation

pip install torch tokenizers

Load Model

import torch
from tokenizers import Tokenizer
from model_v2 import LLaMAModel

device = "cuda" if torch.cuda.is_available() else "cpu"

tokenizer = Tokenizer.from_file("tokenizer.json")
model = LLaMAModel.from_pretrained("K0D3IN/PML-6L").to(device)
model.eval()

Generate Passwords

def generate(model, tokenizer, temperature=0.6, tag=None, n=10):
    prefix_ids = tokenizer.encode(tag).ids if tag else None
    pws = set()
    while len(pws) < n:
        pw = model.generate(
            tokenizer,
            temperature=temperature,
            top_k=50,
            max_len=24,
            min_len=4,
            prefix_ids=prefix_ids,
            device=device,
        )
        if pw:
            pws.add(pw)
    return list(pws)

# Generate 50 passwords for security testing
passwords = generate(model, tokenizer, temperature=0.6, n=50)

Training Details

This model was trained on an NVIDIA GeForce RTX 4060 Laptop GPU (8GB VRAM) using PyTorch 2.11 with CUDA 13.2.

Metric	Value
Training time (4 epochs)	~2 hours 40 minutes
Tokens/second	~123,000
GPU memory usage	~190 MB
Checkpoint size	126 MB

Architecture Diagram (Simplified)

Input IDs → Embedding (vocab=500 → dim=384)
          → RoPE Position Encoding
          → 6× Block:
              ├── RMSNorm
              ├── Causal Self-Attention (6 heads, FlashAttention)
              ├── RMSNorm
              └── SwiGLU MLP (384 → 1024 → 384)
          → RMSNorm
          → LM Head (384 → vocab=500)
          → Logits

Intended Use

✅ Appropriate	❌ Inappropriate
Authorized penetration testing	Credential stuffing against live systems
Red team engagements with written authorization	Account takeover
Password policy research	Illegal access to systems
Security awareness training	Privacy violations
Academic password research	Harassment or doxxing
Internal security audits	Any use without explicit authorization

Ethical Notice

This model generates password candidates that may match real user credentials.

Passwords generated by this model are statistical patterns derived from public breach data (rockyou.txt and sqlmap.txt). They do not contain actual user credentials from the training set. However, due to the nature of password generation, some outputs may coincidentally match real passwords currently in use.

Users are solely responsible for:

• Obtaining proper authorization before any security testing
• Complying with all applicable laws (CFAA, GDPR, KVKK, etc.)
• Ensuring testing is conducted within authorized scope only
• Secure handling and disposal of generated password lists

This model should NOT be used for:

• Any illegal activity
• Attacking systems without explicit written permission
• Mass credential stuffing or account takeover attempts
• Harassing individuals or organizations

Files

File	Size	Description
model.safetensors	~45 MB	Model weights
config.json	~1 KB	HF model configuration
tokenizer.json	~50 KB	BPE tokenizer (vocab=500)
model_v2.py	~8 KB	Model definition code
generation_samples.txt	~10 KB	Example outputs at all temperatures

Citation

@software{PML-6L,
  author = {K0D3IN},
  title = {PML-6L: Password Pattern Generator},
  year = {2026},
  url = {https://huggingface.co/K0D3IN/PML-6L}
}

License

This project is licensed under the MIT License -- see the LICENSE file for details.

Support Open Source AI Research

Every donation helps maintain and improve this project:

Monero (XMR):

83iqXtvVu28ZiL9bsATMerSgbFFiD1J1jc96CcxJLEnAW3KBmBKedWnUAeLvLvEA9aBiUBpHQJs1iNHYtkTLZbNUEymobSS

Bitcoin (BTC):

bc1qmnlvpukcgl0hsr7nje0x8555mhtxjt80wtmlxm