Image-to-Text
Transformers
PyTorch
Safetensors
vision-encoder-decoder
image-text-to-text
vision
nougat
Instructions to use facebook/nougat-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use facebook/nougat-base with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "image-to-text" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # 'pip install "transformers<5.0.0' from transformers import pipeline pipe = pipeline("image-to-text", model="facebook/nougat-base")# Load model directly from transformers import AutoTokenizer, AutoModelForMultimodalLM tokenizer = AutoTokenizer.from_pretrained("facebook/nougat-base") model = AutoModelForMultimodalLM.from_pretrained("facebook/nougat-base") - Notebooks
- Google Colab
- Kaggle
Add figure
#2
by nielsr HF Staff - opened
README.md
CHANGED
|
@@ -17,6 +17,11 @@ Nougat is a [Donut](https://huggingface.co/docs/transformers/model_doc/donut) mo
|
|
| 17 |
|
| 18 |
The model is trained to autoregressively predict the markdown given only the pixels of the PDF image as input.
|
| 19 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
## Intended uses & limitations
|
| 21 |
|
| 22 |
You can use the raw model for transcribing a PDF into Markdown. See the [model hub](https://huggingface.co/models?search=nougat) to look for other
|
|
|
|
| 17 |
|
| 18 |
The model is trained to autoregressively predict the markdown given only the pixels of the PDF image as input.
|
| 19 |
|
| 20 |
+
<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/model_doc/nougat_architecture.jpg"
|
| 21 |
+
alt="drawing" width="600"/>
|
| 22 |
+
|
| 23 |
+
<small> Nougat high-level overview. Taken from the <a href="https://arxiv.org/abs/2308.13418">original paper</a>. </small>
|
| 24 |
+
|
| 25 |
## Intended uses & limitations
|
| 26 |
|
| 27 |
You can use the raw model for transcribing a PDF into Markdown. See the [model hub](https://huggingface.co/models?search=nougat) to look for other
|