Reproducing ParseBench chart score
Hi infly team, congrats on taking #1 on the HuggingFace leaderboard of ParseBench, great work!
We're trying to replay Infinity-Parser2 on run-llama/ParseBench. Using the doc2json prompt from your SDK, text_content, text_formatting, layout, and table come close to the numbers in parsebench.yaml, but chart lands near 0 vs your 61.3. doc2json forces figure text to empty, so chart rules see nothing.
Could you open a PR to run-llama/ParseBench adding the pipelines you used for both Pro and Flash? Once it lands, we'll reproduce and verify the results on our side, then update the ParseBench GitHub leaderboard to reflect your numbers.
Thanks!
Hi,
Thanks for the congratulations and for testing our parser! To achieve the chart parsing score you see on the leaderboard, "deep parsing mode" needs to be enabled for Infinity-Parser2.
We'll open a PR to the run-llama/ParseBench repo very soon. We will include the pipelines for both Pro and Flash to ensure you can reproduce our results.
Looking forward to getting this verified!