Update README.md
Browse files
README.md
CHANGED
|
@@ -59,7 +59,22 @@ let embedding = out.featureValue(for: "embedding")!.multiArrayValue!
|
|
| 59 |
```
|
| 60 |
|
| 61 |
### Rust (objc2-core-ml)
|
| 62 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 63 |
|
| 64 |
### Python (coremltools)
|
| 65 |
```python
|
|
@@ -83,7 +98,7 @@ embedding = out["embedding"] # shape (1, 512)
|
|
| 83 |
5. **Custom `fmod` MIL lowering** — HTSAT's relative-position arithmetic uses float modulo; coremltools has no built-in handler. Registered as `x - trunc(x/y) * y`.
|
| 84 |
6. **`slice_scatter` override** — HTSAT's attention-mask builder generates empty-slice `slice_scatter` calls at deeper Swin stages (e.g. `slice(0, -window_size)` evaluates to `slice(0, 0)`). The built-in handler's shape check rejects these; registered override that no-ops empty slices and reduces non-empty ones to `slice_by_index + concat`.
|
| 85 |
|
| 86 |
-
A full conversion script that applies all six patches is in
|
| 87 |
|
| 88 |
## Validation
|
| 89 |
|
|
|
|
| 59 |
```
|
| 60 |
|
| 61 |
### Rust (objc2-core-ml)
|
| 62 |
+
The `objc2`/`objc2-core-ml` crates give direct Rust bindings to Core ML. Sketch:
|
| 63 |
+
|
| 64 |
+
```rust
|
| 65 |
+
use objc2_core_ml::{MLModel, MLModelConfiguration, MLMultiArray, MLMultiArrayDataType,
|
| 66 |
+
MLDictionaryFeatureProvider, MLFeatureValue, MLComputeUnits};
|
| 67 |
+
|
| 68 |
+
// Core ML wants a compiled .mlmodelc — compile the .mlpackage once,
|
| 69 |
+
// then load with cpuAndGPU compute units.
|
| 70 |
+
let compiled = unsafe { MLModel::compileModelAtURL_error(&mlpackage_url) }?;
|
| 71 |
+
let config = unsafe { MLModelConfiguration::new() };
|
| 72 |
+
unsafe { config.setComputeUnits(MLComputeUnits::CPUAndGPU) };
|
| 73 |
+
let model = unsafe { MLModel::modelWithContentsOfURL_configuration_error(&compiled, &config) }?;
|
| 74 |
+
|
| 75 |
+
// Build [1, 480000] float32 input, copy waveform via dataPointer,
|
| 76 |
+
// wrap in MLFeatureValue + MLDictionaryFeatureProvider, run prediction.
|
| 77 |
+
```
|
| 78 |
|
| 79 |
### Python (coremltools)
|
| 80 |
```python
|
|
|
|
| 98 |
5. **Custom `fmod` MIL lowering** — HTSAT's relative-position arithmetic uses float modulo; coremltools has no built-in handler. Registered as `x - trunc(x/y) * y`.
|
| 99 |
6. **`slice_scatter` override** — HTSAT's attention-mask builder generates empty-slice `slice_scatter` calls at deeper Swin stages (e.g. `slice(0, -window_size)` evaluates to `slice(0, 0)`). The built-in handler's shape check rejects these; registered override that no-ops empty slices and reduces non-empty ones to `slice_by_index + concat`.
|
| 100 |
|
| 101 |
+
A full conversion script that applies all six patches is included in this repo: [`convert-clap-to-coreml.py`](./convert-clap-to-coreml.py). Run with `pip install coremltools>=8,<9 torch>=2.6,<2.10 transformers>=4.40 numpy>=1.24,<2` then `python convert-clap-to-coreml.py --output clap_audio_encoder.mlpackage`. Validation (cosine vs PyTorch reference) runs automatically.
|
| 102 |
|
| 103 |
## Validation
|
| 104 |
|