---
language: en
license: cc-by-sa-4.0
library_name: transformers
pipeline_tag: text-classification
tags:
- natural-language-inference
- monotonicity
- negation
- inoculation
- reproduction
- bert
datasets:
- stanfordnlp/snli
base_model: bert-base-uncased
---

# monli-bert-inoculated

`monli-bert-snli` further fine-tuned ("inoculated") on **NMoNLI** so that it learns
the negation + monotonicity rule, while a mix of SNLI examples preserves general
NLI performance.

This is a **reproduction artifact** — the *Experiment 1.2 model* from our
reproduction of:

> Atticus Geiger, Kyle Richardson, Christopher Potts (2020).
> *Neural Natural Language Inference Models Partially Embed Theories of Lexical
> Entailment and Negation.* BlackboxNLP @ EMNLP 2020. [arXiv:2004.14623](https://arxiv.org/abs/2004.14623)

## Training

Two stages (see base model [`HandongAILab/monli-bert-snli`](https://huggingface.co/HandongAILab/monli-bert-snli) for stage 1):

| Setting | Value |
|---------|-------|
| Base model | `monli-bert-snli` |
| Inoculation data | NMoNLI train (1,002 examples) |
| Mixed-in data | 10K random SNLI examples (prevents catastrophic forgetting) |
| Epochs | 10 (best checkpoint at epoch 4) |
| Learning rate | 5e-6 (low, to preserve SNLI representations) |
| Batch size | 48 |

The low learning rate + SNLI mixing are the key choices that let the model acquire
negation handling without losing SNLI accuracy. The paper does not disclose its
inoculation configuration; this combination reproduces its reported behavior.

## Results

| Dataset | Accuracy |
|---------|----------|
| SNLI test | 90.2% (paper 90.5%) |
| PMoNLI (no negation) | 92.8% (paper: not reported) |
| NMoNLI test (with negation) | 98.0% (paper 90.0%) |

NMoNLI rises from ~3% (baseline) to 98%, and generalizes to **word pairs unseen
during inoculation** — the systematic-generalization result. PMoNLI is *not*
degraded (mild positive cross-transfer), which the paper did not report.

## Labels

| id | label |
|----|-------|
| 0 | entailment |
| 1 | neutral |
| 2 | contradiction |

## Reproduction

Baseline (before inoculation): [`HandongAILab/monli-bert-snli`](https://huggingface.co/HandongAILab/monli-bert-snli)

## Limitations

A reproduction model on a small, templated challenge set (MoNLI). High NMoNLI
accuracy reflects MoNLI's structure and should not be read as general negation
competence. Probing/intervention analyses (see notebook) show the negation rule is
only **partially** localized internally.