Fine-tune for structured extraction
Hello,
Awesome model, I am astonished by the OCR capabilities for such as small model.
In one of the other discussion, you mentioned " One important thing, LightOnOCR is NOT meant to be prompted; adding any input text will just degrade performance.".
Is that something that can be solved with fine-tuning - https://colab.research.google.com/drive/1WjbsFJZ4vOAAlKtcCauFLn_evo5UBRNa?usp=sharing ?
I am looking to perform structured extraction with a given schema and therefore would be interested to prompt the model.
Cheers,
Hi,
Thanks for the feedback!
For the task of OCR only, we trained the model to do the task with no further prompting as the prompt would be something fixed 'Convert this page to Markdown' or some similar variant, it's clean to package when there is no need for a prompt as well.
For structured extraction, it would be interesting to finetune on some (prompt, json-like key-value pairs), there is no reason why it shouldn't work.
Let us know if you try this!