| --- |
| license: mit |
| base_model: facebook/bart-base |
| tags: |
| - text2text-generation |
| - style-transfer |
| - sarcasm |
| - bart |
| - seq2seq |
| language: |
| - en |
| pipeline_tag: summarization |
| --- |
| |
| # BART-Base (Baseline) |
|
|
| Baseline BART-base supervised fine-tuning on sarcastic->non-sarcastic headline pairs. No context enhancement, no RL. |
|
|
| Part of the **Project LLMao** sarcasm style transfer suite (CS4248 Team 14, NUS AY2025/26 S2). |
| This model rewrites sarcastic news headlines as neutral, factual equivalents while |
| preserving the underlying meaning. |
|
|
| ## Task |
|
|
| **Input**: A sarcastic news headline |
| **Output**: A non-sarcastic rewrite |
|
|
| Example: |
| - In: *"Area Man Passionate Defender Of What He Imagines Constitution To Be"* |
| - Out: *"Man defends his interpretation of the Constitution."* |
|
|
| ## Training |
|
|
| - **Base model**: [`facebook/bart-base`](https://huggingface.co/facebook/bart-base) (139M params) |
| - **Method**: Standard cross-entropy on (sarcastic, non-sarcastic) pairs derived from NHDSD. |
| - **Dataset**: 10,868 sarcastic->non-sarcastic headline pairs derived from NHDSD |
| (News Headlines Dataset for Sarcasm Detection). Non-sarcastic targets were generated |
| by an LLM annotator (StepFun Step-3.5 Flash) with cross-validation by Nemotron. |
| Split: `sar_to_non (original)`. |
| - **Input format**: Raw sarcastic headline (no task prefix — BART is not pretrained with prefixes). |
| - **Generation**: beam search with `num_beams=4`, `max_length=128`. |
|
|
| ## Usage |
|
|
| ```python |
| from transformers import AutoTokenizer, AutoModelForSeq2SeqLM |
| |
| model_id = "SeeYangZhi/BART-Base-Sarcasm-Rewriter" |
| tokenizer = AutoTokenizer.from_pretrained(model_id) |
| model = AutoModelForSeq2SeqLM.from_pretrained(model_id) |
| |
| headline = "Area Man Passionate Defender Of What He Imagines Constitution To Be" |
| inputs = tokenizer(headline, return_tensors="pt", truncation=True, max_length=128) |
| outputs = model.generate(**inputs, max_length=128, num_beams=4) |
| print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| ``` |
|
|
| ## Evaluation |
|
|
| Evaluated on a 2,857-sample held-out test split alongside 13 other model variants |
| (BART, T5 baselines, LLaMA 3.2, ablation studies). Metrics include: |
|
|
| | Metric | Direction | |
| |---|---| |
| | Hard Flip Rate (% of samples where sarcasm was removed) | higher ↑ | |
| | Semantic Similarity (all-MiniLM-L6-v2 cosine) | higher ↑ | |
| | BLEU vs input (lower = more genuine rewriting) | lower ↓ | |
| | Perplexity (GPT-2) | lower ↓ | |
| | Normalized edit distance | higher ↑ | |
| | Paraphrase score (low = real rewriting) | lower ↓ | |
|
|
| Full per-variant numbers are published alongside the Project LLMao webapp. |
|
|
| ## Related models |
|
|
| - [`SeeYangZhi/Llama-3.2-1B-Sarcasm-Rewriter`](https://huggingface.co/SeeYangZhi/Llama-3.2-1B-Sarcasm-Rewriter) — instruction-tuned LLaMA variant |
| - `SeeYangZhi/BART-Base-Sarcasm-Rewriter` — supervised baseline |
| - `SeeYangZhi/BART-Base-CE-Sarcasm-Rewriter` — context-enhanced SFT |
| - `SeeYangZhi/BART-Base-RL-Sarcasm-Rewriter` — REINFORCE on top of baseline |
| - `SeeYangZhi/BART-Base-CE-RL-Sarcasm-Rewriter` — CE + RL (best) |
|
|
| ## License |
|
|
| MIT, inheriting from `facebook/bart-base`. The NHDSD dataset is used under its |
| original research-use terms. |
|
|
