melmass/skyseg-coreml

skyseg for Core ML

JianyuanWang/skyseg (a U²-Net sky segmenter) converted to Core ML, fp16, for the Neural Engine. It is used by arrow-swift to skip sky pixels, as ARROW does.

• Input image: float32 [1, 3, 320, 320], RGB in [0, 1] resized to 320 × 320 (bilinear), then normalized with ImageNet's mean (0.485, 0.456, 0.406) and standard deviation (0.229, 0.224, 0.225).
• Output score: [1, 1, 320, 320], the network's final score map (the first of the ONNX model's seven outputs; the side outputs are dropped).

To get a mask as ARROW does: stretch the score map to [0, 1] per frame (min to max), quantize as floor(s · 255) / 255, resize to the image size (bilinear), and call sky where it is at least 32/255.

Conversion

skyseg.onnx → onnx-simplifier → onnx2torch → torch.jit.trace → coremltools (macOS 15 target, fp16). On three real frames with sky, the masks agree with onnxruntime's on 99.98–99.99 % of pixels. On an M1 Pro one frame takes 16.5 ms on the Neural Engine, 39 ms on the GPU and 67 ms on the CPU.

License

MIT, as the original model.