monli-bert-inoculated

monli-bert-snli further fine-tuned ("inoculated") on NMoNLI so that it learns the negation + monotonicity rule, while a mix of SNLI examples preserves general NLI performance.

This is a reproduction artifact — the Experiment 1.2 model from our reproduction of:

Atticus Geiger, Kyle Richardson, Christopher Potts (2020). Neural Natural Language Inference Models Partially Embed Theories of Lexical Entailment and Negation. BlackboxNLP @ EMNLP 2020. arXiv:2004.14623

Training

Two stages (see base model HandongAILab/monli-bert-snli for stage 1):

Setting	Value
Base model	monli-bert-snli
Inoculation data	NMoNLI train (1,002 examples)
Mixed-in data	10K random SNLI examples (prevents catastrophic forgetting)
Epochs	10 (best checkpoint at epoch 4)
Learning rate	5e-6 (low, to preserve SNLI representations)
Batch size	48

The low learning rate + SNLI mixing are the key choices that let the model acquire negation handling without losing SNLI accuracy. The paper does not disclose its inoculation configuration; this combination reproduces its reported behavior.

Results

Dataset	Accuracy
SNLI test	90.2% (paper 90.5%)
PMoNLI (no negation)	92.8% (paper: not reported)
NMoNLI test (with negation)	98.0% (paper 90.0%)

NMoNLI rises from ~3% (baseline) to 98%, and generalizes to word pairs unseen during inoculation — the systematic-generalization result. PMoNLI is not degraded (mild positive cross-transfer), which the paper did not report.

Labels

id	label
0	entailment
1	neutral
2	contradiction

Reproduction

Baseline (before inoculation): HandongAILab/monli-bert-snli

Limitations

A reproduction model on a small, templated challenge set (MoNLI). High NMoNLI accuracy reflects MoNLI's structure and should not be read as general negation competence. Probing/intervention analyses (see notebook) show the negation rule is only partially localized internally.