ai-classifier-small-v4

ai-classifier-small-v4 is a binary sequence classification model fine-tuned to identify artificial intelligence (AI) related statements, requirements, and skills within job postings. This version (v4) represents a significant upgrade over previous iterations, having been further fine-tuned and validated using data sourced across two distinct job postings corpora, to ensure greater domain generalizability.

Basic Usage

You can deploy this model using the standard Hugging Face text-classification pipeline.

from transformers import pipeline, AutoModelForSequenceClassification, AutoTokenizer

model_name = "loyoladatamining/ai-classifier-small-v4"
model = AutoModelForSequenceClassification.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name, max_length=128, truncation=True)

# Create text classification pipeline
nlp = pipeline(
    "text-classification", 
    model=model, 
    tokenizer=tokenizer,
    max_length=128,
    truncation=True
)

# Inference
text = "Experience building large language models (LLMs) or deploying machine learning pipelines is required."
result = nlp(text)
print(result)

Output Format

The model returns a list containing a single classification result with the predicted binary label and its associated confidence score:

[
  {
    "label": "LABEL_1",
    "score": 0.9842
  }
]

Label Mapping

• LABEL_0: The text does not contain statements or requirements related to AI.
• LABEL_1: The text contains explicitly AI-related statements, technologies, or job requirements.

Evaluation

The performance of ai-classifier-small-v4 was evaluated against the previous iteration (ai-classifier-small-v3.1), using the loyoladatamining/usajobs_validation dataset. This newer version demonstrates significantly better performance on the AI statement classification portion:

Model	Accuracy	F-1
ai-classifier-small-v3.1	0.6375	0.6780
ai-classifier-small-v4	0.9339	0.9343

Citation

If you find this model useful in your work, please consider citing:

@article{meisenbacher2025extracting,
  title={Extracting O* NET Features from the NLx Corpus to Build Public Use Aggregate Labor Market Data},
  author={Meisenbacher, Stephen and Nestorov, Svetlozar and Norlander, Peter},
  year={2025}
}