Testing OCR using vllm returns nothing but exclamation marks.
I found the first version of this model to be quite useful so I was excited to try out the newest version to see the improvement in OCR quality, unfortunately I cant get it to work using the provided vllm usage example, whether it's the PDF provided in the test code or my own local files, the OCR output is always just a bunch of "!!!!!!!!!!!!!!!!!" and nothing else.
I found the first version of this model to be quite useful so I was excited to try out the newest version to see the improvement in OCR quality, unfortunately I cant get it to work using the provided vllm usage example, whether it's the PDF provided in the test code or my own local files, the OCR output is always just a bunch of "!!!!!!!!!!!!!!!!!" and nothing else.
Are you sure your using the bf16 settings and not f16? Thats what usually happens when its not correct.
I found the first version of this model to be quite useful so I was excited to try out the newest version to see the improvement in OCR quality, unfortunately I cant get it to work using the provided vllm usage example, whether it's the PDF provided in the test code or my own local files, the OCR output is always just a bunch of "!!!!!!!!!!!!!!!!!" and nothing else.
Are you sure your using the bf16 settings and not f16? Thats what usually happens when its not correct.
I have a Turing GPU so no bf16 support, vllm automatically falls back to f16.
hello,
how about when using transformers? fp16 on T4 with vllm could be a vllm issue
hello,
how about when using transformers? fp16 on T4 with vllm could be a vllm issue
Using transformers does work but is obviously much much slower, I guess the issue might be with vllm even tho I can still run the first model just fine.
I found the first version of this model to be quite useful so I was excited to try out the newest version to see the improvement in OCR quality, unfortunately I cant get it to work using the provided vllm usage example, whether it's the PDF provided in the test code or my own local files, the OCR output is always just a bunch of "!!!!!!!!!!!!!!!!!" and nothing else.
I also encountered the same problem.
I found the first version of this model to be quite useful so I was excited to try out the newest version to see the improvement in OCR quality, unfortunately I cant get it to work using the provided vllm usage example, whether it's the PDF provided in the test code or my own local files, the OCR output is always just a bunch of "!!!!!!!!!!!!!!!!!" and nothing else.
I also encountered the same problem.
What GPU are you using ?
is it true for all the LightOnOCR-2 variants?
is it true for all the LightOnOCR-2 variants?
I tried ocr-soup and bbox variant to check and yes, same result on all 3 variants. I also tried to use other attention backends in vllm like 'TRITON_ATTN' and 'FLEX_ATTENTION' to no avail.
thats weird as we have used vLLM for all our evaluations and the demo is running on vLLM too for the first two models. is it the case for all samples or just a particular one?