File size: 8,391 Bytes
d82db76 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 | # PaddleOCR-Compatible Recognizer Training on Apple MPS
Status: verified one-step training evidence on `mbp2` and `k2` as of
2026-08-09.
This project can train the PP-OCRv5 mobile recognizer's CTC path on Apple GPUs.
It does so with a small PyTorch adapter that preserves the PaddleOCR model
architecture, not with the stock PaddlePaddle trainer. That distinction matters:
the data, dictionary, input profile, and architecture remain PaddleOCR-shaped,
but the resulting checkpoint is a PyTorch checkpoint and cannot be loaded into
PaddleOCR as a `.pdparams` file.
## Decision
| Concern | Current path | Evidence status |
| --- | --- | --- |
| Recognizer model and optimizer | PyTorch MPS | Forward, backward, update, and checkpoint verified |
| CTC log probabilities | PyTorch MPS | Verified |
| CTC loss | CPU with autograd back to MPS | Verified; this is an intentional device boundary |
| Official PaddleOCR training program | Stock PaddlePaddle | No stock MPS device on the tested runtime |
| Paddle's custom MPS plugin | `PaddleCustomDevice` prototype | Not yet sufficient for PP-OCRv5 training |
| Checkpoint format | PyTorch `state_dict` | Adapter-only until a converter and parity test exist |
Use this path for recognizer experiments on the available Macs. Keep the
existing Paddle-first page pipeline and its data/evaluation contracts. If an
official Paddle checkpoint is a delivery requirement, train on a supported
Paddle device or build and validate a deliberate checkpoint converter.
## Why Stock Paddle Is Not the Immediate MPS Path
A direct probe on `k2` with PaddlePaddle 3.3.1 returned no custom devices and
rejected `mps`, `metal`, and `custom_device:metal` device selections. This
matches Paddle's [macOS CPU build documentation][paddle-macos] and the
[PaddleOCR recognition training guide][paddleocr-rec], which does not support
macOS GPU training through the stock runtime. The documented
[`paddle.device.set_device` forms][paddle-device] also do not include a stock
MPS target.
Paddle does have an official [MPS custom-device backend][paddle-mps]. Its
current build identifies itself as version `0.0.1`, requires a signed local
build, and its kernel tree covers only a small operator subset such as
activation, elementwise, matmul, reshape, softmax, squeeze, and unsqueeze. Its
[README][paddle-mps-readme] proves the prototype can expose an `mps` device and
run a simple softmax. It does not yet establish the convolution, normalization,
sequence, loss, optimizer, and autograd coverage required by PP-OCRv5. Treat it
as the longer-term native-Paddle route, not the current training backend.
## The Practical Adapter
The vendored subset comes from the Apache-2.0
[`frotms/PaddleOCR2Pytorch` revision `c05307f`][adapter-source]. It contains the
PP-OCRv5 mobile recognition components needed here:
- `PPLCNetV3` at scale `0.95`;
- the SVTR encoder in the MultiHead CTC branch;
- the CTC head used for this training path.
The NRTR auxiliary branch is not part of the current loss. Its parameters are
frozen in the verified smoke run. This is therefore a PP-OCRv5-compatible CTC
training path, not a claim of complete reproduction of PaddleOCR's multi-loss
recipe.
Three local changes make dynamic-width MPS training work:
1. The backbone pools height to one while retaining recognition width instead
of forcing the original fixed training width. The verified time-axis rule is
`T = input_width / 4`: width 160 produces 40 CTC steps and width 320 produces
80.
2. A localized contiguity hook before
`ctc_encoder.encoder.conv3` prevents the MPS convolution from receiving the
non-contiguous layout emitted by the preceding SVTR operations.
3. The model, images, SVTR/CTC logits, and log-softmax remain on MPS. Only
`CTCLoss` runs on CPU; PyTorch autograd carries its gradient back to the MPS
graph, after which the optimizer updates MPS parameters.
The CPU loss boundary is a correctness solution, not a throughput claim. It
adds device transfers and still needs multi-epoch profiling on real manifests.
## Verified Smoke Evidence
The direct checks below exercised the same model shape and completed a finite
gradient update. Their elapsed values include cold-start effects and are
diagnostics, **not benchmarks**.
| Host | Runtime | Input -> logits | Loss | Tracked max parameter delta | Cold elapsed |
| --- | --- | --- | ---: | ---: | ---: |
| `mbp2` (M4 Pro, 20-core GPU, 24 GB) | Torch 2.13, MPS | `[1,3,48,320] -> [1,80,48]` | about 55.91 | `0.000501692` | about 3.96 s |
| `k2` (M2 Ultra, 60-core GPU, 64 GB) | Torch 2.12, MPS | `[1,3,48,320] -> [1,80,48]` | `55.94594` | `0.000501603` | `9.871` s |
An end-to-end adapter smoke on `mbp2` used Python 3.11.15 and Torch 2.13.0.
Four synthetic samples covered Latin, Devanagari, and Limbu characters. Input
`[4,3,48,160]` produced MPS logits `[4,40,7]`; CPU CTC loss was
`29.883989334106445`; all 582 gradient tensors were finite on `mps:0`; and an
optimizer step changed the tracked parameter by `0.000500023365020752`. The
step took 7.3767 seconds and the sampled MPS current allocation after the step
was 226,097,408 bytes. Neither number is a sustained-throughput or peak-memory
measurement.
The ephemeral evidence artifacts on `mbp2` were:
- `/tmp/paddle2torch-mps-smoke/artifacts/result.json`, SHA-256
`44427429b01ef308a386bc2cbbab415ee11e78a09262213cee7d0c600ffd1c48`;
- `/tmp/paddle2torch-mps-smoke/artifacts/ppocrv5_mobile_ctc_mps_step.pt`,
SHA-256
`0113bb40a024bd88609c529a51b39a88f55f80a8cb4173d110382be162f148ca`.
`/tmp` is not durable storage. A real run must write its config, manifest
fingerprint, source revision, environment report, metrics, and checkpoint to a
versioned run directory.
## Checkpoint Compatibility Boundary
The adapter checkpoint records PyTorch tensor names, optimizer state, and the
script-specific CTC head. Do not rename it to `.pdparams` or pass it to the
official PaddleOCR trainer. Likewise, a Paddle checkpoint is not a direct warm
start for this path merely because the architecture names match.
Any conversion must explicitly map parameter names and layouts, handle the
dictionary-dependent CTC output size, document the local dynamic-pooling
change, and prove inference parity on a frozen input set. Until that work
exists, compatibility means shared architecture intent, input geometry,
dictionary, and datasets—not shared checkpoint files or runtime behavior.
## Using `mbp2` and `k2`
The two Macs improve experiment throughput by running independent jobs. They
should not be treated as one distributed MPS training job: this project has no
validated multi-host MPS DDP path.
- Use immutable dataset/config fingerprints on both hosts, but give every job
a distinct run directory, seed, and experiment ID.
- Use `mbp2` for smoke runs and focused ablations. Its 24 GB unified memory and
existing memory pressure make conservative batch sizes important.
- Use `k2` for memory-heavier or longer candidates; its 64 GB unified memory is
the more useful capacity advantage.
- Collect checkpoints and canonical prediction files after each independent
run, then compare them with `tools/eval_metrics.py`. Paddle
`norm_edit_dis` is not CER, and silver labels remain diagnostic rather than
claim-grade evidence.
GPU access is useful now and the available MPS machines are sufficient to start
architecture and training experiments. The immediate proof still needed is a
multi-epoch real-manifest run with checkpoint resume/reload, stable memory,
canonical held-out CER, and comparison against the existing Paddle-trained
recognizers. An NVIDIA GPU is not required for this adapter path; it remains
useful only when native Paddle training or CUDA-specific parity is required.
[adapter-source]: https://github.com/frotms/PaddleOCR2Pytorch/tree/c05307fbd61e575d55f0fef0334026c7536aec76
[paddle-device]: https://www.paddlepaddle.org.cn/documentation/api/paddle/device/set_device_cn.html
[paddle-macos]: https://www.paddlepaddle.org.cn/documentation/docs/en/install/compile/macos-compile-ninja.html
[paddle-mps]: https://github.com/PaddlePaddle/PaddleCustomDevice/tree/develop/backends/mps
[paddle-mps-readme]: https://github.com/PaddlePaddle/PaddleCustomDevice/blob/develop/backends/mps/README.md
[paddleocr-rec]: https://github.com/PaddlePaddle/PaddleOCR/blob/main/docs/version2.x/ppocr/model_train/recognition.en.md
|