sennatitcomb/sarcasm-detector-json-only-final

Contextual Sarcasm Detector (Fine-Tuned RoBERTa)

Model Description

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-irony designed to detect sarcasm in narrative and conversational contexts.

Key Research Improvements

Unlike standard irony detectors trained on short-form social media (e.g., the Joshi Dataset), this model was fine-tuned on a consolidated pool of high-signal samples. The Joshi dataset was explicitly removed during experimentation to reduce noise and prevent "classifier paranoia" in modern conversational contexts.

Training Data

The model was trained on a balanced corpus of 152 samples:

• Class 1 (Sarcastic): Curated contextual JSON samples focusing on literary and modern narrative and conversational irony.
• Class 0 (Sincere): Curated contextual JSON samples focusing on literary modern dialogue and sentences.

Evaluation Results

The fine-tuned model demonstrated a significant architectural improvement over the baseline irony model.

Metric	Base Irony Model	Fine-Tuned Model
Golden Set F1	0.5714	0.8889
Golden Set Acc	0.7000	0.9000
Human Set F1	0.5455	0.6667
Human Set Acc	0.5000	0.7000

Research Findings & Limitations

Domain Shift and Syntax Bias

Qualitative analysis via a sliding-window inference test revealed a Syntax-Based Decision Boundary.

1. Modern Success: The model successfully calibrated its probabilities in modern dialogue, distinguishing between narrative setup and ironic punchlines.
2. Literary Regression: A performance regression was noted in the "Mr. Collins" test case. Because the model's "Sincere" (Class 0) training data was heavily derived from Victorian prose, it developed a heuristic where formal/classical syntax is strongly correlated with sincerity. This resulted in a failure to detect biting irony when expressed in a formal style.

Intended Use

This model is best suited for modern narrative text or conversational AI agents requiring context-aware sarcasm detection. It is less effective on formal literary irony from the 19th century.