piyushptiwari/insureos-models

INSUREOS Models — Complete Insurance AI Training Pipeline

Created by Bytical AI — AI agents that run insurance operations.

Overview

INSUREOS is a complete AI/ML training and inference pipeline for UK insurance operations. This repository contains all source code for data generation, model training, evaluation, data collection, and a hybrid search engine.

Model Suite

Model	HuggingFace	Task	Key Metric
InsureLLM-4B	piyushptiwari/InsureLLM-4B	Insurance domain LLM	ROUGE-1: 0.384
InsureDocClassifier	piyushptiwari/InsureDocClassifier	12-class document classification	F1: 1.0
InsureNER	piyushptiwari/InsureNER	13-entity NER	F1: 1.0
InsureFraudNet	piyushptiwari/InsureFraudNet	Fraud detection (3 LoB)	AUC-ROC: 1.0
InsurePricing	piyushptiwari/InsurePricing	Premium pricing (GLM + EBM)	MAE: £11,132
InsureSearch	(included in this repo)	Hybrid search engine	33K docs indexed

Training Dataset

piyushptiwari/insureos-training-data — 10K SFT, 5K DPO, 50K tabular, 10K docs, 8K NER

Repository Structure

insureos-models/
├── data/                        # Synthetic data generation
│   ├── constants.py             # UK insurance constants (regions, perils, regulators)
│   ├── gen_sft.py               # Generate SFT instruction-response pairs
│   ├── gen_dpo.py               # Generate DPO preference pairs
│   ├── gen_documents.py         # Generate insurance documents (12 classes)
│   ├── gen_ner.py               # Generate NER-annotated text
│   ├── gen_tabular.py           # Generate claims tabular data
│   └── generate_all.py          # Run all generators
│
├── collect/                     # Real-world data collection
│   ├── config.py                # Scraping targets and configuration
│   ├── scraper_base.py          # Base HTTP scraper with caching
│   ├── convert_sft.py           # Convert raw docs → SFT/DPO format
│   ├── run_fast.py              # Fast collection orchestrator
│   └── sources/                 # Per-source scrapers
│       ├── wikipedia.py         # Wikipedia insurance articles
│       ├── legislation.py       # UK legislation (legislation.gov.uk)
│       ├── fca.py               # FCA Handbook
│       ├── hf_datasets.py       # HuggingFace insurance datasets
│       ├── rss_news.py          # Insurance news RSS feeds
│       └── education.py         # Insurance education resources
│
├── training/                    # Model training scripts
│   ├── qlora_finetune.py        # QLoRA fine-tuning (Qwen3-4B)
│   ├── dpo_train.py             # DPO alignment training
│   ├── retrain_realworld.py     # Real-world data retraining
│   ├── doc_classifier.py        # ModernBERT document classifier
│   ├── ner_model.py             # ModernBERT NER model
│   ├── fraud_model.py           # XGBoost + Isolation Forest fraud
│   ├── pricing_glm.py           # Tweedie GLM + EBM pricing
│   └── distill.py               # Model distillation (experimental)
│
├── evaluation/                  # Evaluation suite
│   ├── run_eval.py              # Full multi-model evaluation
│   └── results/                 # Evaluation results (JSON)
│
├── search/                      # Hybrid search engine
│   ├── config.py                # Search configuration
│   ├── embedder.py              # BGE-small-en-v1.5 embedding service
│   ├── bm25.py                  # Custom Okapi BM25 implementation
│   ├── vector_store.py          # Qdrant vector store
│   ├── reranker.py              # Cross-encoder reranker
│   ├── hybrid_engine.py         # RRF fusion (vector + BM25 + reranker)
│   ├── indexer.py               # Document ingestion pipeline
│   ├── models.py                # Pydantic data models
│   └── api.py                   # FastAPI REST API
│
├── serve/                       # Model serving
│   └── api.py                   # FastAPI inference endpoints
│
└── scripts/                     # Automation
    ├── setup.sh                 # Environment setup (NVIDIA, Python, deps)
    └── train_all.sh             # Full training pipeline script

Quick Start

1. Environment Setup

# Create virtual environment
python3 -m venv .venv && source .venv/bin/activate

# Install dependencies
pip install torch transformers trl peft bitsandbytes
pip install xgboost scikit-learn interpret
pip install sentence-transformers qdrant-client fastapi uvicorn

2. Generate Training Data

python -m data.generate_all
# Outputs: data/output/ (SFT, DPO, docs, NER, tabular)

3. Train Models

# Train all models sequentially
bash scripts/train_all.sh

# Or individually:
python training/qlora_finetune.py          # InsureLLM QLoRA
python training/dpo_train.py               # InsureLLM DPO
python training/doc_classifier.py          # Document classifier
python training/ner_model.py               # NER model
python training/fraud_model.py             # Fraud detection
python training/pricing_glm.py             # Pricing models

4. Evaluate

python evaluation/run_eval.py
# Results saved to evaluation/results/

5. Run Search Engine

# Index documents
python search/indexer.py

# Start API
python search/api.py
# API at http://localhost:8900
# Endpoints: /search, /search/vector, /search/keyword, /suggest, /facets, /stats

Search Engine — InsureSearch

A hybrid search engine rivaling Azure AI Search, built entirely on open-source components:

Component	Technology	Details
Vector Search	BGE-small-en-v1.5 (384-dim) + Qdrant	Semantic similarity
Keyword Search	Custom Okapi BM25	Insurance-aware tokenization
Reranking	cross-encoder/ms-marco-MiniLM-L-6-v2	Cross-encoder reranking
Fusion	Reciprocal Rank Fusion (RRF)	Vector 60% + BM25 40%
API	FastAPI	REST API with facets, suggestions

Index stats: 33,034 chunks from 31,679 documents, 51,640 BM25 terms.

Training Pipeline

Stage 1: Synthetic Data Generation
├── 10K SFT instruction-response pairs
├── 5K DPO preference pairs
├── 50K tabular claims (Motor/Property/Liability)
├── 10K insurance documents (12 classes)
└── 8K NER-annotated texts (13 entity types)

Stage 2: QLoRA Fine-Tuning → Qwen3-4B
├── rank=64, alpha=128, all-linear targets
├── 2 epochs, batch=2, grad_accum=4
├── Final: train_loss=0.012, eval_loss=0.118
└── Token accuracy: 95.88%

Stage 3: DPO Alignment
├── 5K preference pairs
├── 149 steps, reward_accuracy=1.0
└── Reward margin: 26.76

Stage 4: Real-World Data Collection
├── Wikipedia (150 docs), UK Legislation (692)
├── HuggingFace datasets (31,060), RSS (50), Education (88)
├── Converted to 3,685 SFT + 776 DPO pairs
└── Quality filtered (English-only, no echo responses)

Stage 5: Real-World Retraining
├── 876 steps on real-world SFT data
└── Claims process score improved 0.40 → 0.60

Stage 6: Specialized Models (parallel)
├── FraudNet: XGBoost + Isolation Forest → AUC-ROC 1.0
├── PricingGLM: Tweedie GLM + EBM → MAE £11,132
├── DocClassifier: ModernBERT → F1 1.0
└── InsureNER: ModernBERT → F1 1.0

Tech Stack

• LLM: Qwen3-4B + QLoRA + DPO (PyTorch, Transformers, TRL, PEFT, bitsandbytes)
• Classification & NER: ModernBERT-base (Transformers)
• Fraud Detection: XGBoost + Isolation Forest (scikit-learn)
• Pricing: Tweedie GLM (scikit-learn) + EBM (InterpretML)
• Search: BGE-small-en-v1.5 + Qdrant + BM25 + cross-encoder
• Training GPU: NVIDIA Tesla T4 16GB

Citation

@misc{bytical2026insureos,
  title={INSUREOS: A Complete AI/ML Suite for UK Insurance Operations},
  author={Bytical AI},
  year={2026},
  url={https://huggingface.co/piyushptiwari/insureos-models}
}

About Bytical AI

Bytical builds AI agents that run insurance operations — claims automation, underwriting intelligence, digital sales, and core system modernization for insurers across the UK and Europe. Microsoft AI Partner | NVIDIA | Salesforce.