add handler.py for HF Dedicated Inference
#14
by jmbrito - opened
This PR adds a Inference Handler to this model.
This is required for using it in a HuggingFace Dedicated Endpoint, since this product does not has a Image-To-Text task available out-of-the-box.
The handler implements the inference as described in the model docs but with the StoppingCriteria implemented on the notebook example.
Hi,
Note that this isn't required to be added to this repo. The only thing you need is defining a custom_handler.py in a new HF model repo which uses:
from transformers import VisionEncoderDecoderModel
model = VisionEncoderDecoderModel.from_pretrained("facebook/nougat-base")
, no need to push anything to this repo.
See this blog post as an example: https://www.philschmid.de/stable-diffusion-controlnet-endpoint.