--- license: cc-by-sa-4.0 language: - en base_model: facebook/bart-base # No explicit pipeline_tag: "text2text-generation" is rejected by the Hub's current # validation list (it renders a warning banner and mislabels the model as decoder-only # "Text Generation"), and none of the accepted tags fit sentence simplification -- # "summarization" is wrong for a roughly length-preserving rewrite. Left unset so the # Hub infers it from config.json's BartForConditionalGeneration instead. tags: - text-simplification - accessibility - bart datasets: - wikilarge - facebook/asset metrics: - sari - bleu model-index: - name: bart-base-wikilarge-simplification results: - task: type: text2text-generation name: Sentence-level text simplification dataset: name: ASSET (test split) type: facebook/asset metrics: - type: sari value: 37.80 - type: bleu value: 67.26 - type: fkgl value: 8.18 name: FKGL (lower is better) --- # bart-base-wikilarge-simplification `facebook/bart-base` fine-tuned on **WikiLarge** for **sentence-level** English text simplification. It rewrites one sentence at a time into simpler English while preserving meaning. Trained as part of a bachelor thesis on automated simplification of everyday English web text, and served as the `local` model by the project's backend: ## Intended use Simplifying a **single English sentence** of everyday prose. The companion project uses it behind a browser extension that sentence-splits page text (`pysbd`) and sends each sentence independently. **Out of scope:** multi-sentence input, whole documents, languages other than English, and any setting where a factual error would be harmful. This is a research artifact from a student project — it hallucinates (see Limitations) and has had no human evaluation. For whole-document rewriting, use the companion model [`yunvs/bart-base-dwikipedia-simplification`](https://huggingface.co/yunvs/bart-base-dwikipedia-simplification) instead. ## Usage ```python from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tok = AutoTokenizer.from_pretrained("yunvs/bart-base-wikilarge-simplification") model = AutoModelForSeq2SeqLM.from_pretrained("yunvs/bart-base-wikilarge-simplification") text = "Despite the multiplicity of antecedent conditions, the framework mandates uniform compliance." inputs = tok(text, return_tensors="pt", truncation=True, max_length=64) out = model.generate(**inputs, max_length=64, num_beams=4, no_repeat_ngram_size=3, repetition_penalty=1.2) print(tok.decode(out[0], skip_special_tokens=True)) ``` **Use `max_length=64`.** That is the length the model was fine-tuned at, and generating at BART's 512/1024 default degrades output. Input past 64 tokens is truncated, not chunked — split into sentences first. ## Training | | | |---|---| | Base model | `facebook/bart-base` | | Dataset | WikiLarge, ~117k training pairs after filtering corrupted rows | | Epochs | 5 (best checkpoint selected at epoch 4) | | Batch size | 16 | | Learning rate | 3e-5 | | Weight decay | 0.01 | | Max length | 64 tokens (WikiLarge p90 ≈ 52) | | Seed | 42 | | Selection | `load_best_model_at_end`, `metric_for_best_model="loss"`, early stopping (patience 4 quarter-epoch evals) | | Final validation loss | 0.4568 | Early stopping did not halt the run; epoch 5 (val loss 0.4590) did not improve on epoch 4, and `load_best_model_at_end` selected epoch 4 correctly. ## Evaluation Evaluated on the **ASSET** test split (359 sentences, 10 human references each) with beam search (`num_beams=4`), against the un-fine-tuned `facebook/bart-base` as a zero-shot baseline. | | SARI ↑ | BLEU | FKGL ↓ | |---|---|---|---| | **This model** | **37.80** | 67.26 | **8.18** | | `facebook/bart-base` zero-shot | 21.34 | 89.89 | 10.02 | Paired bootstrap significance test on the SARI improvement, 1000 resamples: **p < 0.001**. ### Reading the BLEU number **BLEU going the "wrong way" here is expected, not a contradiction.** BLEU is biased toward conservative, low-edit output, and ASSET's 10 references per sentence amplify that: an output that stays close to the source (as the zero-shot baseline largely does) has many more chances to match *some* reference's n-grams, inflating BLEU without the output being a good simplification. SARI — which rewards appropriate edits against the source — and FKGL both favour the fine-tuned model by a wide margin (+16.5 SARI, −1.85 grade levels). Treat BLEU here as a documented counterexample of its known limitation for simplification, not as evidence the baseline is better. ## Limitations - **Hallucination.** Short or fragmentary inputs can produce unrelated text; the phrase `"Other websites"` (a Simple English Wikipedia section heading present in the training data) has been observed as output for several unrelated short inputs. The companion backend runs a quality guard that falls back to the original text in this case. - **Conservative rewriting.** Observed output often drops a subordinate clause rather than rephrasing dense vocabulary — it does not reliably simplify words like "multiplicity" or "heterogeneous entities." - **No human evaluation.** All reported numbers are automatic metrics. No readability, meaning-preservation, or accessibility study has been conducted. - **Single sentences only.** Multi-sentence input is out of distribution and gets truncated at 64 tokens. - **English only**, and trained on Wikipedia prose — encyclopedic register, with whatever demographic and topical bias that corpus carries. ## Training data provenance and licence WikiLarge is derived from English Wikipedia and Simple English Wikipedia, which are licensed **CC BY-SA**. This model is released under **CC BY-SA 4.0** accordingly. WikiSmall was evaluated as an alternative and deliberately rejected: it carries baked-in named-entity anonymization artifacts unsuitable for training. ## Citation Produced for a bachelor thesis (2026). Please cite the project repository: