Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
nchhayaย 
posted an update 2 days ago
Post
2,088
Stamps on top of the fields you need are where most document AI breaks. We released 10 open synthetic benchmarks for exactly these failure cases: stamped documents, handwriting, bad scans, dense pages, PII redaction, splitting and more, each with answer keys and the generator code (CC BY 4.0).

Try HyperAPI on them in the live demo: nchhaya/hyperapi-demo

How to benchmark any pipeline on them: https://huggingface.co/blog/nchhaya/benchmark-document-ai-synthetic-datasets

I have seen dense text break a lot of models that otherwise do really well on benchmarks ... excited to test models out on these benchmarks!

In this post