Instructions to use PaddlePaddle/PaddleOCR-VL with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PaddleOCR
How to use PaddlePaddle/PaddleOCR-VL with PaddleOCR:
# See https://www.paddleocr.ai/latest/version3.x/pipeline_usage/PaddleOCR-VL.html to installation from paddleocr import PaddleOCRVL pipeline = PaddleOCRVL(pipeline_version="v1") output = pipeline.predict("path/to/document_image.png") for res in output: res.print() res.save_to_json(save_path="output") res.save_to_markdown(save_path="output") - Notebooks
- Google Colab
- Kaggle
Keep broadcast mask dims in per-image vision attention; pass BlockMask through
Browse files- Per-image eager/SDPA vision attention sliced both mask dims, so a broadcastable `[B, 1, 1, L]` padding mask got an empty query dim for every image after the first and failed with a size mismatch (e.g. `cu_seqlens=[0, 4, 8]`). Size-1 dims are now kept and only real sequence dims are sliced.
- `_create_causal_mask` read `attention_mask.ndim`, which a flex-attention `BlockMask` does not have. The dense 4D conversion now applies only to `torch.Tensor`, so a BlockMask reaches the native `create_causal_mask`, which returns it unchanged.
Tested with transformers 4.57.6, 5.12.1 and 5.22.1: packed attention with `[B,1,1,L]` and `[B,1,L,L]` padding masks matches running each image alone (eager, SDPA), a BlockMask is returned as the same object, and the existing CPU fp32 / A100 bf16 (eager, SDPA, FA2) regression checks still pass.
Co-Authored-By: Claude Code <noreply@anthropic.com>
- modeling_paddleocr_vl.py +11 -5
|
@@ -104,7 +104,8 @@ def _init_tensor(tensor, operation, *args, **kwargs):
|
|
| 104 |
|
| 105 |
|
| 106 |
def _create_causal_mask(config, inputs_embeds, attention_mask, past_key_values, cache_position):
|
| 107 |
-
|
|
|
|
| 108 |
mask = attention_mask.to(device=inputs_embeds.device)
|
| 109 |
if mask.dtype == torch.bool:
|
| 110 |
additive = torch.zeros(mask.shape, device=mask.device, dtype=inputs_embeds.dtype)
|
|
@@ -1241,10 +1242,15 @@ class PaddleOCRAttention(nn.Module):
|
|
| 1241 |
if output_attentions else None
|
| 1242 |
)
|
| 1243 |
for start, end in zip(boundaries[:-1], boundaries[1:]):
|
| 1244 |
-
local_mask =
|
| 1245 |
-
|
| 1246 |
-
|
| 1247 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1248 |
output, weights = attention_interface(
|
| 1249 |
self, queries[:, :, start:end], keys[:, :, start:end],
|
| 1250 |
values[:, :, start:end], local_mask,
|
|
|
|
| 104 |
|
| 105 |
|
| 106 |
def _create_causal_mask(config, inputs_embeds, attention_mask, past_key_values, cache_position):
|
| 107 |
+
# Only dense tensors need conversion; flex BlockMask is passed through to the native function.
|
| 108 |
+
if isinstance(attention_mask, torch.Tensor) and attention_mask.ndim == 4:
|
| 109 |
mask = attention_mask.to(device=inputs_embeds.device)
|
| 110 |
if mask.dtype == torch.bool:
|
| 111 |
additive = torch.zeros(mask.shape, device=mask.device, dtype=inputs_embeds.dtype)
|
|
|
|
| 1242 |
if output_attentions else None
|
| 1243 |
)
|
| 1244 |
for start, end in zip(boundaries[:-1], boundaries[1:]):
|
| 1245 |
+
local_mask = None
|
| 1246 |
+
if attention_mask is not None:
|
| 1247 |
+
# Keep size-1 broadcast dims, e.g. a [B, 1, 1, L] padding mask.
|
| 1248 |
+
span = slice(start, end)
|
| 1249 |
+
local_mask = attention_mask[
|
| 1250 |
+
...,
|
| 1251 |
+
span if attention_mask.shape[-2] > 1 else slice(None),
|
| 1252 |
+
span if attention_mask.shape[-1] > 1 else slice(None),
|
| 1253 |
+
]
|
| 1254 |
output, weights = attention_interface(
|
| 1255 |
self, queries[:, :, start:end], keys[:, :, start:end],
|
| 1256 |
values[:, :, start:end], local_mask,
|