Add model card

README.md

@@ -0,0 +1,127 @@
+---
+license: cc-by-4.0
+language:
+- es
+pipeline_tag: text-classification
+library_name: sklearn
+datasets:
+- Flaglab/ESNLIR-dataset
+metrics:
+- accuracy
+- f1
+tags:
+- nli
+- spanish
+- causal
+- esnlir
+- xgboost
+- baseline
+---
+
+# ESNLIR — XGBoost baseline
+
+The **non-neural reference point** for ESNLIR: XGBoost over bag-of-words features from both
+sentences. It exists to show how much of the task is solvable without contextual representations —
+the answer is *not much*, which is what makes the BERT results meaningful.
+
+> **ESNLIR: Expanding Spanish NLI Benchmarks with Multi-Genre and Causal Annotation**
+> Johan R. Portela, Nicolás Pérez-Terán, Rubén Manrique — Universidad de los Andes, Bogotá
+> *Applied Informatics*, Springer, 2026, pp. 345–361 —
+> [doi:10.1007/978-3-032-07175-0_23](https://doi.org/10.1007/978-3-032-07175-0_23)
+
+Part of the [ESNLIR collection](https://huggingface.co/collections/Flaglab/esnlir-6914a3103bb229f3c3f09567).
+
+## Results on the ESNLIR test set (80,216 pairs, class-balanced)
+
+| accuracy | macro F1 | contrasting | entailment | neutral | reasoning |
+|---|---|---|---|---|---|
+| **0.3501** | **0.3481** | 0.345 | 0.436 | 0.278 | 0.341 |
+
+Ten points above the 0.250 majority-class floor, and roughly **half** what the fine-tuned encoders
+reach:
+
+| model | accuracy | macro F1 |
+|---|---|---|
+| Majority class | 0.250 | 0.250 |
+| **XGBoost (this model)** | **0.350** | **0.348** |
+| [BERTIN RoBERTa](https://huggingface.co/Flaglab/ESNLIR-RoBERTa) | 0.663 | 0.664 |
+| [XLM-RoBERTa](https://huggingface.co/Flaglab/ESNLIR-XLM-RoBERTa) | 0.676 | 0.676 |
+
+Lexical features alone barely move the needle, so the relations in ESNLIR are not recoverable from
+word identity — contextual sentence representations are doing the work.
+
+### Stress tests
+
+| test | accuracy | macro F1 | contrasting | entailment | neutral | reasoning |
+|---|---|---|---|---|---|---|
+| `test` | 0.3501 | 0.3481 | 0.345 | 0.436 | 0.278 | 0.341 |
+| `test_length_mismatch` | 0.3257 | 0.2945 | 0.204 | 0.307 | 0.111 | 0.681 |
+| `test_negation` | 0.3381 | 0.3287 | 0.401 | 0.266 | 0.200 | 0.486 |
+| `test_overlap` | 0.3409 | 0.3313 | 0.226 | 0.360 | 0.245 | 0.532 |
+| `test_spelling` | 0.3490 | 0.3473 | 0.340 | 0.428 | 0.280 | 0.348 |
+
+The perturbations push predictions hard toward `reasoning` (0.341 → 0.681 under length mismatch),
+which is what a bag-of-words model does when tokens are injected: the added words shift the feature
+vector rather than the meaning.
+
+## Labels
+
+| id | label |
+|---|---|
+| 0 | `contrasting` |
+| 1 | `entailment` |
+| 2 | `neutral` |
+| 3 | `reasoning` |
+
+Label order is alphabetical, from `sklearn.preprocessing.LabelEncoder`.
+
+## Files
+
+`model.pkl` — a pickled scikit-learn/XGBoost estimator. `metrics.zip` — full metrics per split,
+broken down by genre, domain and source corpus.
+
+## Usage
+
+```python
+import pickle
+from huggingface_hub import hf_hub_download
+
+with open(hf_hub_download("Flaglab/ESNLIR-Baseline", "model.pkl"), "rb") as fh:
+    model = pickle.load(fh)
+```
+
+> Loading a pickle executes arbitrary code — only do this because you trust the source.
+>
+> The vectorizer is **not** included in this repository, so the model cannot be applied to raw text
+> as published. Rebuild the bag-of-words features with
+> [`jd-rodriguezp1234/esnlir`](https://github.com/jd-rodriguezp1234/esnlir)
+> (`auto_nli/model/baseline/dataset.py`), which fits the vectorizer and label encoder on the
+> training split. For inference on new text, prefer
+> [`ESNLIR-XLM-RoBERTa`](https://huggingface.co/Flaglab/ESNLIR-XLM-RoBERTa).
+
+## Training
+
+| | |
+|---|---|
+| features | bag-of-words over both sentences |
+| data | [`Flaglab/ESNLIR-dataset`](https://huggingface.co/datasets/Flaglab/ESNLIR-dataset) |
+| max samples | 1,000,000 (subsampled from the 4.4M train split) |
+
+Trained with `auto_nli/model/baseline/run.py`, config `params/model/baseline.json`.
+
+## Citation
+
+```bibtex
+@InProceedings{portela2025esnlirspanishmultigenredataset,
+  author    = {Portela, Johan R. and P{\'e}rez-Ter{\'a}n, Nicol{\'a}s and Manrique, Rub{\'e}n},
+  editor    = {Florez, Hector and Peluffo-Ordo{\~{n}}ez, Diego},
+  title     = {{ESNLIR}: Expanding Spanish {NLI} Benchmarks with Multi-genre and Causal Annotation},
+  booktitle = {Applied Informatics},
+  year      = {2026},
+  publisher = {Springer Nature Switzerland},
+  address   = {Cham},
+  pages     = {345--361},
+  isbn      = {978-3-032-07175-0},
+  doi       = {10.1007/978-3-032-07175-0_23},
+}
+```