README.md

---
license: mit
base_model: microsoft/mdeberta-v3-base
tags:
- causal-extraction
- causality
- cause-effect
- span-extraction
- causal-news-corpus
- multilingual
language:
- en
- es
- fr
- de
- pt
- tr
- ru
- ar
- zh
- ja
---

# causal-span-pointer-v2

A **span-pointer** causal extraction model: given a sentence it predicts the
**cause**, **effect** and **signal** spans as start/end pointers, decoded under
ordering/non-overlap constraints with beam search (top-2 relations per sentence).
Fine-tuned from [`microsoft/mdeberta-v3-base`](https://huggingface.co/microsoft/mdeberta-v3-base)
on the [Causal News Corpus](https://github.com/tanfiona/CausalNewsCorpus) Subtask-2
(CC0), augmented with synthetic cause/effect data in 6 languages (en, es, fr, de, nl, tr)
and hard negatives that train the causal gate. Architecture reimplemented from the CNC
baseline (MIT).

## Benchmark (Causal News Corpus Subtask 2)

Official scorer (`evaluation/subtask2`: FairEval + best-combination alignment), V2 dev:

```
Overall   Precision 0.708   Recall 0.694   F1 0.699
Cause     F1 0.726
Effect    F1 0.702
Signal    F1 0.661

Causal gate precision (940 multilingual negatives): 0.999
Synthetic test per role, 6 languages: Cause 0.975 / Effect 0.972 / Signal 0.946
```

**This beats the organizer's 0.627 dev baseline** and the 2022 shared-task winner
(0.542, test); it trails the 2023 winner (0.728, test). Same scorer, same dev set.

### vs a few-shot LLM

A prompted general LLM does not match this fine-tune. Qwen2.5-7B-Instruct, few-shot on
the same dev set and official scorer, scores **0.24** F1 with a plain causal prompt and
**0.41** with a scheme-aware prompt (vs **0.70** here). Even on a capability-fair subset
-- causality a reader recognises without CNC's broad purpose/motive/implicit
conventions -- the LLM reaches ~0.45 vs this model's ~0.63. The residual gap is exact
span-boundary precision, which fine-tuning on the annotation provides.

## Usage

This is a custom architecture, so inference goes through the `causal_span_model`
package (not `AutoModel`):

```python
from huggingface_hub import snapshot_download
from causal_span_model.pointer.submission import load_pointer, predict_sentence

local_dir = snapshot_download("Berk/causal-span-pointer-v2")
model, tokenizer = load_pointer(local_dir)
print(predict_sentence(model, tokenizer, "Heavy rainfall caused severe flooding."))
# ['<ARG0>Heavy rainfall</ARG0> <SIG0>caused</SIG0> <ARG1>severe flooding</ARG1> .', ...]
```

`<ARG0>` = cause, `<ARG1>` = effect, `<SIG0>` = signal. The prediction is a list of
tagged relation strings (up to two per sentence).

### Multilingual

Trained on English CNC spans plus synthetic cause/effect data in 6 languages, and
multilingual at inference (mDeBERTa encoder + script-aware segmentation). Use
`predict_relations`, which returns character-exact
spans in any script:

```python
from causal_span_model.pointer.infer import predict_relations

predict_relations(model, tokenizer, "暴雨导致该地区发生严重洪灾。")
# [{'cause': '暴雨', 'effect': '该地区发生严重洪灾', 'signal': '导致'}]
predict_relations(model, tokenizer, "Las fuertes lluvias provocaron inundaciones.")
# [{'cause': 'Las fuertes lluvias', 'effect': 'inundaciones', 'signal': 'provocaron'}]
```

Verified on es/fr/de/pt/tr/ru/ar and CJK (zh/ja).

## Notes

- It is NOT compatible with a generic token-classification ONNX consumer -- it
  needs its own start/end + beam-search decoder (provided by the package).
- It has a built-in **causal gate** (a causal/non-causal head, ~0.85 accuracy on
  CNC dev): `predict_relations` returns `[]` on text it judges non-causal, so it
  is safe to run on arbitrary input. Beam duplicates are collapsed to one relation
  per distinct cause->effect.

## Companion causal gate (`token_gate/`)

The repo also ships a fine-tuned **token-aware causal gate** in `token_gate/`: a
`paraphrase-multilingual-MiniLM-L12-v2` sequence classifier (P(causal); id2label
`{0: non_causal, 1: causal}`) that decides whether a sentence expresses a causal relation
before the pointer extracts spans. Unlike a frozen-embedding gate it keys on the relation, not
the topic, so it separates a verb-causal sentence from its plain twin ("The GPU cluster
increased training throughput" -> 0.99 vs "The GPU cluster is installed in rack 4" -> 0.01).

reasongraph >= 0.7.1:

```python
from reasongraph import CausalPointerExtractor
ex = CausalPointerExtractor(
    model="Berk/causal-span-pointer-v2",
    token_gate="hf://Berk/causal-span-pointer-v2/token_gate",
    token_gate_threshold=0.10)
```

Or directly:

```python
import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer
tok = AutoTokenizer.from_pretrained("Berk/causal-span-pointer-v2", subfolder="token_gate")
m = AutoModelForSequenceClassification.from_pretrained("Berk/causal-span-pointer-v2", subfolder="token_gate")
enc = tok("It flooded because it rained.", return_tensors="pt")
p_causal = torch.softmax(m(**enc).logits, -1)[0, 1].item()  # ~0.98
```

At cutoff **0.10** on the 39-case reviewed set / 60 plain facts: keeps 75% of causal hop
facts, rejects **100%** of plain facts, synthetic-dev F1 0.98, ~5 ms/sentence on CPU. It is
stricter than the companion embedding gate (`embed_gate_mlp.joblib`, which keeps ~92% of hop
facts but lets ~25% of plain facts through): use the token gate when clean plain-fact
rejection matters, the embedding gate for maximum causal recall.

## License

MIT (weights and code). Training data is CC0-1.0.