Model Card for TRPaliGemma

This model is fine-tuned PaliGemma model for the Table recognition task.

Model Details

Model Description

Table recognition is a branch of Document AI. In the existing Table recognition, the structure of the table and the OCR results were calculated and combined, respectively. For this reason, unnecessary predictions are sometimes made in the process of parsing the table.(ex. bbox) Using VLM, the structure and text of the table will be predicted at the same time, eliminating unnecessary predictions and integrating the two tasks into one.

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

• Developed by: Seokhyun Choi
• Funded by [optional]: [More Information Needed]
• Shared by [optional]: [More Information Needed]
• Model type: Vision Language Model
• Language(s) (NLP): English
• License: [More Information Needed]
• Finetuned from model [optional]: PaliGemma

Model Sources [optional]

• Repository: [More Information Needed]
• Paper [optional]: [More Information Needed]
• Demo [optional]: [More Information Needed]

Uses

Direct Use

This model can convert a tabular images into HTML.

Downstream Use [optional]

It can be used in document automation systems using Document AI.

Out-of-Scope Use

This is a fine-tuned model with only the tabular images that exist within the PDF, so you won't get good performance in the tabular images in the wild.

Bias, Risks, and Limitations

This model simply converts table images into HTML. To gain additional analysis or knowledge, you need to learn an NLP model for analysis using HTML or fine-tune the new PaliGemma model by constructing new data.

How to Get Started with the Model

inference : https://www.kaggle.com/code/mldlchoidh/tr-inference

Training Details

Training Data

Pubtables1-1M