LoveJesus/evangelism-intent-classifier-chirho

Evangelism Intent Classifier (RoBERTa-base)

Part of Model 9: Evangelism & Apologetics Pipeline for bible.systems.

Model Description

A fine-tuned RoBERTa-base model that classifies user questions into 5 intent categories for routing in the evangelism/apologetics pipeline:

Intent	Description	Test F1
evangelism_dialogue	Direct evangelism conversations	0.62
apologetics_qa	Apologetics questions & answers	0.99
creation_science	Creation science & intelligent design	0.71
historical_evidence	Historical evidence for Christianity	0.89
miracle_testimony	Miracle testimonies & documentation	0.94

Performance

• Test Macro F1: 0.8303
• Test Weighted F1: 0.9810
• Test Loss: 0.7690
• Training: 5 epochs, RoBERTa-base with WeightedTrainer (inverse-frequency class weights)

Pipeline Architecture

User Question -> [Intent Classifier] -> route to appropriate handler
    |-> evangelism_dialogue -> Generator directly
    |-> apologetics_qa -> Retriever -> RAG -> Generator
    |-> creation_science -> Retriever -> RAG -> Generator
    |-> historical_evidence -> Retriever -> RAG -> Generator
    |-> miracle_testimony -> Retriever -> RAG -> Generator

Usage

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

model = AutoModelForSequenceClassification.from_pretrained("LoveJesus/evangelism-intent-classifier-chirho")
tokenizer = AutoTokenizer.from_pretrained("LoveJesus/evangelism-intent-classifier-chirho")

labels = ["evangelism_dialogue", "apologetics_qa", "creation_science", "historical_evidence", "miracle_testimony"]

text = "What evidence is there for the resurrection of Jesus?"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)

with torch.no_grad():
    logits = model(**inputs).logits
    pred = torch.argmax(logits, dim=-1).item()

print(f"Intent: {labels[pred]}")  # -> apologetics_qa

Training Data

10,476 training examples from diverse apologetics sources including GotQuestions.org, Spurgeon sermons, early church fathers, creation science evidence, historical evidence compilations, and miracle testimonies.

Related Models

• LoveJesus/evangelism-retriever-chirho - Passage retriever
• LoveJesus/evangelism-generator-chirho - Response generator (Qwen3-14B LoRA)
• LoveJesus/evangelism-dataset-chirho - Training dataset