language: en
license: cc-by-sa-4.0
library_name: transformers
pipeline_tag: text-classification
tags:
- natural-language-inference
- monotonicity
- negation
- inoculation
- reproduction
- bert
datasets:
- stanfordnlp/snli
base_model: bert-base-uncased
monli-bert-inoculated
monli-bert-snli further fine-tuned ("inoculated") on NMoNLI so that it learns
the negation + monotonicity rule, while a mix of SNLI examples preserves general
NLI performance.
This is a reproduction artifact — the Experiment 1.2 model from our reproduction of:
Atticus Geiger, Kyle Richardson, Christopher Potts (2020). Neural Natural Language Inference Models Partially Embed Theories of Lexical Entailment and Negation. BlackboxNLP @ EMNLP 2020. arXiv:2004.14623
Training
Two stages (see base model HandongAILab/monli-bert-snli for stage 1):
| Setting | Value |
|---|---|
| Base model | monli-bert-snli |
| Inoculation data | NMoNLI train (1,002 examples) |
| Mixed-in data | 10K random SNLI examples (prevents catastrophic forgetting) |
| Epochs | 10 (best checkpoint at epoch 4) |
| Learning rate | 5e-6 (low, to preserve SNLI representations) |
| Batch size | 48 |
The low learning rate + SNLI mixing are the key choices that let the model acquire negation handling without losing SNLI accuracy. The paper does not disclose its inoculation configuration; this combination reproduces its reported behavior.
Results
| Dataset | Accuracy |
|---|---|
| SNLI test | 90.2% (paper 90.5%) |
| PMoNLI (no negation) | 92.8% (paper: not reported) |
| NMoNLI test (with negation) | 98.0% (paper 90.0%) |
NMoNLI rises from ~3% (baseline) to 98%, and generalizes to word pairs unseen during inoculation — the systematic-generalization result. PMoNLI is not degraded (mild positive cross-transfer), which the paper did not report.
Labels
| id | label |
|---|---|
| 0 | entailment |
| 1 | neutral |
| 2 | contradiction |
Reproduction
Baseline (before inoculation): HandongAILab/monli-bert-snli
Limitations
A reproduction model on a small, templated challenge set (MoNLI). High NMoNLI accuracy reflects MoNLI's structure and should not be read as general negation competence. Probing/intervention analyses (see notebook) show the negation rule is only partially localized internally.