| | --- |
| | library_name: transformers |
| | tags: [] |
| | --- |
| | |
| | # Model Card for Model ID |
| |
|
| | <!-- Provide a quick summary of what the model is/does. --> |
| |
|
| |
|
| |
|
| | ## Model Details |
| |
|
| | ### Model Description |
| |
|
| | <!-- Provide a longer summary of what this model is. --> |
| |
|
| | This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated. |
| |
|
| | - **Developed by:** Sefika Efeoglu |
| |
|
| | - **Model type:** text-to-text |
| | - **Language(s) (NLP):** [More Information Needed] |
| | - **License:** [More Information Needed] |
| | - **Finetuned from model [optional]:** https://huggingface.co/google/flan-t5-base |
| |
|
| |
|
| | ## Uses |
| |
|
| | ```python |
| | import json |
| | import torch |
| | from transformers import AutoTokenizer |
| | from transformers import AutoModelForCausalLM |
| | from datetime import datetime |
| | from transformers import T5Tokenizer, T5ForConditionalGeneration |
| | |
| | tokenizer = T5Tokenizer.from_pretrained("google/flan-t5-base") |
| | model_id = "Sefika/semeval_base_1" |
| | model = T5ForConditionalGeneration.from_pretrained(model_id, |
| | device_map="auto", |
| | load_in_8bit=False, |
| | torch_dtype=torch.float16) |
| | prompt = """Example Sentence:The purpose of the <e1>audit</e1> was to report on the <e2>financial statements</e2>.\n"""+\ |
| | """Sentence: Query Sentence:The most common <e1>audits</e1> were about <e2>waste</e2> and recycling.\n"""+\ |
| | """What is the relation type between e1: audits. and e2 : waste. according to given relation types below in the sentence?\n"""+\ |
| | """Relation types: Relation types: Cause-Effect(e2,e1), Content-Container(e1,e2), Member-Collection(e1,e2), Instrument-Agency(e1,e2), Product-Producer(e2,e1), Member-Collection(e2,e1), Message-Topic(e1,e2), Entity-Origin(e2,e1), Message-Topic(e2,e1), Instrument-Agency(e2,e1), Content-Container(e2,e1), Product-Producer(e1,e2), Entity-Origin(e1,e2), Component-Whole(e1,e2), Entity-Destination(e1,e2), Other, Cause-Effect(e1,e2), Component-Whole(e2,e1), Entity-Destination(e2,e1). \n""" |
| | inputs = self.tokenizer(prompt, add_special_tokens=True, max_length=526,return_tensors="pt").input_ids.to("cuda") |
| | |
| | outputs = self.model.generate(inputs, max_new_tokens=length, pad_token_id=self.tokenizer.eos_token_id) |
| | |
| | response = self.tokenizer.batch_decode(outputs, skip_special_tokens=True) |
| | print(response[0]) |
| | #"Cause-Effect(e1,e2)" |
| | |
| | ``` |
| |
|
| |
|
| | ## Training Details |
| |
|
| | ### Training Data |
| |
|
| | semeval-2010-task8 |
| | [More Information Needed] |
| |
|
| | ### Training Procedure |
| |
|
| | 5 fold cross validation with sentence and relation types. Input is sentence and the output is relation types |
| |
|
| | #### Training Hyperparameters |
| |
|
| | Epoch:5, BS:16 and others are default. |
| |
|
| | #### Hardware |
| |
|
| | Colab Pro+ A100. |
| |
|
| |
|
| |
|
| | ## Citation |
| |
|
| | Efeoglu, Sefika, and Adrian Paschke. "Retrieval-Augmented Generation-based Relation Extraction." arXiv preprint arXiv:2404.13397 (2024). |
| | https://www.semantic-web-journal.net/system/files/swj3810.pdf |
| |
|
