{
    "model_id": "Babelscape/t5-base-summarization-claim-extractor",
    "downloads": 631466,
    "tags": [
        "transformers",
        "safetensors",
        "t5",
        "text2text-generation",
        "en",
        "arxiv:2403.02270",
        "license:cc-by-nc-sa-4.0",
        "autotrain_compatible",
        "text-generation-inference",
        "endpoints_compatible",
        "region:us"
    ],
    "description": "--- library_name: transformers language: - en license: - cc-by-nc-sa-4.0 widget: - text: \"A major tech company has unveiled its first fully autonomous electric vehicle, boasting a range of 500 miles per charge and advanced safety features designed to revolutionize the transportation industry.\" - text: \"A new global initiative to clean up ocean plastic aims to remove 50% of floating debris within a decade, using innovative autonomous vessels powered by renewable energy.\" - text: \"A historic peace agreement was signed between two long-standing rival nations, marking a turning point in diplomatic relations and promising economic and social cooperation for years to come.\" --- # Model Card: T5-base-summarization-claim-extractor ## Model Description **Model Name:** T5-base-summarization-claim-extractor **Authors:** Alessandro Scirè, Karim Ghonim, and Roberto Navigli **Contact:** scire@diag.uniroma1.it, scire@babelscape.com **Language:** English **Primary Use:** Extraction of atomic claims from a summary ### Overview The T5-base-summarization-claim-extractor is a model developed for the task of extracting atomic claims from summaries. The model is based on the T5 architecture which is then fine-tuned specifically for claim extraction. This model was introduced as part of the research presented in the paper \"FENICE: Factuality Evaluation of summarization based on Natural Language Inference and Claim Extraction\" by Alessandro Scirè, Karim Ghonim, and Roberto Navigli. FENICE leverages Natural Language Inference (NLI) and Claim Extraction to evaluate the factuality of summaries. ArXiv version. ### Intended Use This model is designed to: - Extract atomic claims from summaries. - Serve as a component in pipelines for factuality evaluation of summaries. ## Example Code **Note**: The model outputs the claims in a single string. **Kindly remember to split the string into sentences** in order to retrieve the singular claims. ### Training For details regarding the training process, please checkout our paper( (section 4.1). ### Performance | <div style=\"width: 250px\">Model</div> | easiness<sub>P</sub> | easiness<sub>R</sub> | easiness<sub>F1</sub> | |:-------------------------------------:|:--------------------:|:--------------------:|:---------------------:| | GPT-3.5 | 80.1 | 70.9 | 74.9 | | t5-base-summarization-claim-extractor | 79.2 | 68.8 | 73.4 | **Table 1:** Easiness Precision (easiness<sub>P</sub>), Recall (easiness<sub>R</sub>), and F1 score (easiness<sub>F1</sub>) results for the LLM-based claim extractor, namely **GPT-3.5**, and **t5-base-summarization-claim-extractor**, assessed on ROSE (Liu et al., 2023b). Further details on the model's performance and the metrics used can be found in the paper (section 4.1). ### Main Repository For more details about FENICE, check out the GitHub repository: Babelscape/FENICE ### Citation If you use this model in your work, please cite the following paper: ### Limitations - The model is specifically designed for extracting claims from summaries and may not perform well on other types of texts. - The model is currently available only in English and may not generalize well to other languages. ### Ethical Considerations Users should be aware that while this model extracts claims that can be evaluated for factuality, it does not determine the truthfulness of those claims. Therefore, it should be used in conjunction with other tools or human judgment when evaluating the reliability of summaries. ### Acknowledgments This work was made possible thanks to the support of Babelscape and Sapienza NLP.",
    "model_explanation_gemini": "Extracts atomic claims from summaries to support factuality evaluation pipelines."
}