atan2f commited on
Commit
0fe658c
·
verified ·
1 Parent(s): 922371c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -2
README.md CHANGED
@@ -59,7 +59,22 @@ let embedding = out.featureValue(for: "embedding")!.multiArrayValue!
59
  ```
60
 
61
  ### Rust (objc2-core-ml)
62
- See [`scripts/convert-clap-to-coreml.py`](#how-it-was-built) reference repo for a Rust integration that loads this `.mlpackage`, runs sliding-window embedding for arbitrary-length audio, and mean-pools.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
63
 
64
  ### Python (coremltools)
65
  ```python
@@ -83,7 +98,7 @@ embedding = out["embedding"] # shape (1, 512)
83
  5. **Custom `fmod` MIL lowering** — HTSAT's relative-position arithmetic uses float modulo; coremltools has no built-in handler. Registered as `x - trunc(x/y) * y`.
84
  6. **`slice_scatter` override** — HTSAT's attention-mask builder generates empty-slice `slice_scatter` calls at deeper Swin stages (e.g. `slice(0, -window_size)` evaluates to `slice(0, 0)`). The built-in handler's shape check rejects these; registered override that no-ops empty slices and reduces non-empty ones to `slice_by_index + concat`.
85
 
86
- A full conversion script that applies all six patches is in the reference repo (link below).
87
 
88
  ## Validation
89
 
 
59
  ```
60
 
61
  ### Rust (objc2-core-ml)
62
+ The `objc2`/`objc2-core-ml` crates give direct Rust bindings to Core ML. Sketch:
63
+
64
+ ```rust
65
+ use objc2_core_ml::{MLModel, MLModelConfiguration, MLMultiArray, MLMultiArrayDataType,
66
+ MLDictionaryFeatureProvider, MLFeatureValue, MLComputeUnits};
67
+
68
+ // Core ML wants a compiled .mlmodelc — compile the .mlpackage once,
69
+ // then load with cpuAndGPU compute units.
70
+ let compiled = unsafe { MLModel::compileModelAtURL_error(&mlpackage_url) }?;
71
+ let config = unsafe { MLModelConfiguration::new() };
72
+ unsafe { config.setComputeUnits(MLComputeUnits::CPUAndGPU) };
73
+ let model = unsafe { MLModel::modelWithContentsOfURL_configuration_error(&compiled, &config) }?;
74
+
75
+ // Build [1, 480000] float32 input, copy waveform via dataPointer,
76
+ // wrap in MLFeatureValue + MLDictionaryFeatureProvider, run prediction.
77
+ ```
78
 
79
  ### Python (coremltools)
80
  ```python
 
98
  5. **Custom `fmod` MIL lowering** — HTSAT's relative-position arithmetic uses float modulo; coremltools has no built-in handler. Registered as `x - trunc(x/y) * y`.
99
  6. **`slice_scatter` override** — HTSAT's attention-mask builder generates empty-slice `slice_scatter` calls at deeper Swin stages (e.g. `slice(0, -window_size)` evaluates to `slice(0, 0)`). The built-in handler's shape check rejects these; registered override that no-ops empty slices and reduces non-empty ones to `slice_by_index + concat`.
100
 
101
+ A full conversion script that applies all six patches is included in this repo: [`convert-clap-to-coreml.py`](./convert-clap-to-coreml.py). Run with `pip install coremltools>=8,<9 torch>=2.6,<2.10 transformers>=4.40 numpy>=1.24,<2` then `python convert-clap-to-coreml.py --output clap_audio_encoder.mlpackage`. Validation (cosine vs PyTorch reference) runs automatically.
102
 
103
  ## Validation
104