|
|
--- |
|
|
license: mit |
|
|
language: |
|
|
- en |
|
|
tags: |
|
|
- statement-extraction |
|
|
- named-entity-recognition |
|
|
- t5 |
|
|
- gemma |
|
|
- seq2seq |
|
|
- nlp |
|
|
- information-extraction |
|
|
- corp-o-rate |
|
|
pipeline_tag: text2text-generation |
|
|
--- |
|
|
|
|
|
# Statement Extractor (T5-Gemma 2) |
|
|
|
|
|
A fine-tuned T5-Gemma 2 model for extracting structured statements from text. Part of [corp-o-rate.com](https://corp-o-rate.com). |
|
|
|
|
|
## Model Description |
|
|
|
|
|
This model extracts subject-predicate-object triples from unstructured text, with automatic entity type recognition and coreference resolution. |
|
|
|
|
|
- **Architecture**: T5-Gemma 2 (270M-270M, 540M total parameters) |
|
|
- **Training Data**: 77,515 examples from corporate and news documents |
|
|
- **Final Eval Loss**: 0.209 |
|
|
- **Max Input Length**: 4,096 tokens |
|
|
- **Max Output Length**: 2,048 tokens |
|
|
|
|
|
## Usage |
|
|
|
|
|
### Python |
|
|
|
|
|
```python |
|
|
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer |
|
|
import torch |
|
|
|
|
|
model = AutoModelForSeq2SeqLM.from_pretrained( |
|
|
"Corp-o-Rate-Community/statement-extractor", |
|
|
torch_dtype=torch.bfloat16, |
|
|
trust_remote_code=True, |
|
|
) |
|
|
tokenizer = AutoTokenizer.from_pretrained( |
|
|
"Corp-o-Rate-Community/statement-extractor", |
|
|
trust_remote_code=True, |
|
|
) |
|
|
|
|
|
text = "Apple Inc. announced a commitment to carbon neutrality by 2030." |
|
|
inputs = tokenizer(f"<page>{text}</page>", return_tensors="pt") |
|
|
outputs = model.generate(**inputs, max_new_tokens=2048, num_beams=4) |
|
|
result = tokenizer.decode(outputs[0], skip_special_tokens=True) |
|
|
print(result) |
|
|
``` |
|
|
|
|
|
### Input Format |
|
|
|
|
|
Wrap your text in `<page>` tags: |
|
|
|
|
|
``` |
|
|
<page>Your text here...</page> |
|
|
``` |
|
|
|
|
|
### Output Format |
|
|
|
|
|
The model outputs XML with extracted statements: |
|
|
|
|
|
```xml |
|
|
<statements> |
|
|
<stmt> |
|
|
<subject type="ORG">Apple Inc.</subject> |
|
|
<object type="EVENT">carbon neutrality by 2030</object> |
|
|
<predicate>committed to</predicate> |
|
|
<text>Apple Inc. committed to achieving carbon neutrality by 2030.</text> |
|
|
</stmt> |
|
|
</statements> |
|
|
``` |
|
|
|
|
|
## Entity Types |
|
|
|
|
|
| Type | Description | |
|
|
|------|-------------| |
|
|
| ORG | Organizations (companies, agencies) | |
|
|
| PERSON | People (names, titles) | |
|
|
| GPE | Geopolitical entities (countries, cities) | |
|
|
| LOC | Locations (mountains, rivers) | |
|
|
| PRODUCT | Products (devices, services) | |
|
|
| EVENT | Events (announcements, meetings) | |
|
|
| WORK_OF_ART | Creative works (reports, books) | |
|
|
| LAW | Legal documents | |
|
|
| DATE | Dates and time periods | |
|
|
| MONEY | Monetary values | |
|
|
| PERCENT | Percentages | |
|
|
| QUANTITY | Quantities and measurements | |
|
|
|
|
|
## Demo |
|
|
|
|
|
Try the interactive demo at [statement-extractor.vercel.app](https://statement-extractor.vercel.app) |
|
|
|
|
|
## Training |
|
|
|
|
|
- Base model: `google/t5gemma-2-270m-270m` |
|
|
- Training examples: 77,515 |
|
|
- Final eval loss: 0.209 |
|
|
- Training with refinement phase (LR=1e-6, epochs=0.2) |
|
|
- Beam search: num_beams=4 |
|
|
|
|
|
## About corp-o-rate |
|
|
|
|
|
This model is part of [corp-o-rate.com](https://corp-o-rate.com) - an AI-powered platform for ESG analysis and corporate accountability. |
|
|
|
|
|
## License |
|
|
|
|
|
MIT License |
|
|
|
