---
license: gpl-2.0
language: en
base_model: google-bert/bert-base-uncased
pipeline_tag: token-classification
tags:
- causal-extraction
- relation-extraction
- bert
- pytorch
- causality
library_name: transformers
---

# JointCausalModel for Causal Extraction

This repository contains JointCausalModel, a PyTorch-based model for joint causal extraction, optimized for use with the Hugging Face transformers library. The model is built upon `google-bert/bert-base-uncased` and is designed to identify and structure causal relationships within text.

**GitHub Repository**: [https://github.com/rasoulnorouzi/JointLearning](https://github.com/rasoulnorouzi/JointLearning/tree/main)

## Model Description

This model performs three tasks simultaneously:

1. **Sentence-level Causal Classification**: Determines whether a sentence contains a causal statement.
2. **Span Extraction**: Identifies the specific Cause, Effect, and combined Cause-Effect spans within the text using a BIO tagging scheme.
3. **Relation Extraction**: Establishes the relationships between the identified cause and effect spans.

> **Note**: This model uses a custom implementation and requires `trust_remote_code=True` when loading with AutoModel.

## How to Use

To get started, load the model and tokenizer from the Hugging Face Hub:

```python
from transformers import AutoModel, AutoTokenizer

repo_id = "rasoultilburg/SocioCausaNet"

model = AutoModel.from_pretrained(
    repo_id,
    trust_remote_code=True
)

tokenizer = AutoTokenizer.from_pretrained(
    repo_id
)
```

### Inference API

The primary method for inference is `model.predict()`, which processes a list of sentences and returns detailed causal information:

```python
# Example of a simple prediction call
results = model.predict(
    sents=["The heavy rainfall led to severe flooding in the coastal regions."],
    tokenizer=tokenizer
)
```

### Understanding the predict() Parameters

Think of this model as a "Causality Detective." The parameters are the instructions you give the detective on how to investigate the text.

| Parameter | What it is & How it works | Analogy |
|-----------|---------------------------|---------|
| `sents` | The list of sentences you want the model to analyze. | The "case files" you give to the detective. |
| `rel_mode` | Strategy for finding relationships.<br/>- `'auto'`: A smart, efficient mode. For simple cases (one cause-one effect, one cause-multiple effects, multiple causes-one effect), it automatically connects them using rules. For complex cases (multiple causes and multiple effects), it uses a neural network to determine connections.<br/>- `'neural_only'`: Uses a neural network to validate every potential cause-effect connection, checking whether there is a relationship between each pair of entities. More thorough but slower. | The Detective's Strategy<br/>- `'auto'` is the Smart Detective who uses simple logic for obvious cases but calls in the expert (neural network) for complex situations.<br/>- `'neural_only'` is the Expert Detective who carefully analyzes every possible connection using advanced techniques (neural network) regardless of complexity. |
| `rel_threshold` | The confidence score needed to report a relationship (from 0.0 to 1.0).<br/>- High value (e.g., 0.8): Only reports relationships it's very sure about. Fewer, but more accurate results.<br/>- Low value (e.g., 0.3): Reports any potential link, even hunches. More results, but some may be incorrect. | The Detective's "Burden of Proof."<br/>- High value: Needs a lot of evidence before making an accusation.<br/>- Low value: Follows up on even the smallest lead. |
| `cause_decision` | The criteria for deciding if a sentence is causal.<br/>- `'cls_only'`: Decides based on overall sentence meaning.<br/>- `'span_only'`: Decides only if it finds distinct "cause" and "effect" phrases.<br/>- `'cls+span'`: Strictest mode. Sentence must have causal meaning AND contain distinct cause/effect phrases. | The Panel of Judges<br/>- `'cls_only'` is the "Big Picture" Judge.<br/>- `'span_only'` is the "Evidence-Focused" Judge.<br/>- `'cls+span'` requires both judges to agree. Most reliable option. |

## Complete Example

Here is a complete, runnable example demonstrating how to use the model and format the output:

```python
from transformers import AutoModel, AutoTokenizer
import json

# 1. Load the model and tokenizer
repo_id = "rasoultilburg/SocioCausaNet"
model = AutoModel.from_pretrained(repo_id, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(repo_id)

# 2. Define input sentences
sentences = [
    "Insomnia causes depression and a lack of concentration in children.",
    "Due to the new regulations, the company's profits declined sharply.",
    "The sun rises in the east."  # Non-causal example
]

# 3. Get predictions from the model
results = model.predict(
    sentences,
    tokenizer=tokenizer,
    rel_mode="auto",
    rel_threshold=0.5,
    cause_decision="cls+span"
)

# 4. Print the results in a readable format
print(json.dumps(results, indent=2, ensure_ascii=False))
```

### Example Output

The predict method returns a list of dictionaries, where each dictionary corresponds to an input sentence:

```json
[
  {
    "text": "Insomnia causes depression and a lack of concentration in children.",
    "causal": true,
    "relations": [
      {
        "cause": "Insomnia",
        "effect": "depression",
        "type": "Rel_CE"
      },
      {
        "cause": "Insomnia",
        "effect": "a lack of concentration in children",
        "type": "Rel_CE"
      }
    ]
  },
  {
    "text": "Due to the new regulations, the company's profits declined sharply.",
    "causal": true,
    "relations": [
      {
        "cause": "the new regulations",
        "effect": "the company's profits declined sharply",
        "type": "Rel_CE"
      }
    ]
  },
  {
    "text": "The sun rises in the east.",
    "causal": false,
    "relations": [],
    "spans": []
  }
]
```

## Model Architecture

The JointCausalModel requires custom code, which is why `trust_remote_code=True` is necessary. The architecture consists of a BERT encoder followed by three specialized heads for the joint tasks.

The key files defining the model are:

- `modeling_joint_causal.py`: Contains the main JointCausalModel class which defines the model's architecture. It inherits from `transformers.PreTrainedModel` to ensure compatibility with the Hugging Face ecosystem.
- `configuration_joint_causal.py`: Defines the JointCausalConfig class, which stores the model's configuration and hyperparameters.

## Citation

If you use this model in your work, please consider citing this repository.

```bibtex
@misc{jointcausalmodel2024,
  title={JointCausalModel: Joint Learning for Causal Extraction},
  author={Rasoul Norouzi},
  year={2024},
  howpublished={GitHub Repository},
  url={https://github.com/rasoulnorouzi/JointLearning/tree/main}
}
```

For more details and source code, visit the [GitHub repository](https://github.com/rasoulnorouzi/JointLearning/tree/main)