al8n commited on
Commit
a144c29
·
verified ·
1 Parent(s): afce987

fp16-safe conversion: fused x-logsumexp(x) tail; encoder weights byte-identical to upstream pin

Browse files
CHECKSUMS.sha256 ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ d07ca6d8c69f2921810ca9bc919f0334f78de40a8f0c50e125358765fc5769fd ./base960h_aligner.mlmodelc/analytics/coremldata.bin
2
+ 64b5214b50e003435d6659b8a3e4178e58ba48f3738daca0d6a1101412cb5a55 ./base960h_aligner.mlmodelc/coremldata.bin
3
+ b7e9bf36c9290d843804f4082b5442098fd14d26f2e0b032225ddebe3df729b2 ./base960h_aligner.mlmodelc/metadata.json
4
+ 2d55c09bd32ee0810214d3896d0a5e87b2e8643b77c034bfcc908836e10d3809 ./base960h_aligner.mlmodelc/model.mil
5
+ de51193fe73fb3aad085f9c794f08bfde1b939fc12f92e0834edcd4cb712e642 ./base960h_aligner.mlmodelc/weights/weight.bin
6
+ ff4b7c3781d2a07698cb8883050697a45abb9e21325da3d9254df35139553c9c ./base960h_aligner.mlpackage/Data/com.apple.CoreML/model.mlmodel
7
+ de51193fe73fb3aad085f9c794f08bfde1b939fc12f92e0834edcd4cb712e642 ./base960h_aligner.mlpackage/Data/com.apple.CoreML/weights/weight.bin
8
+ bae000742a392d10914bce5d02bcb86ba4bbda83de8c7a141d27c79035fab0d3 ./base960h_aligner.mlpackage/Manifest.json
9
+ e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 ./CHECKSUMS.sha256
README.md ADDED
@@ -0,0 +1,60 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - coreml
5
+ - forced-alignment
6
+ - wav2vec2
7
+ - ctc
8
+ - apple-neural-engine
9
+ library_name: coremlit
10
+ ---
11
+
12
+ # alignkit-coreml — fp16-safe CoreML conversion of wav2vec2-base-960h for forced alignment
13
+
14
+ A CoreML conversion of **facebook/wav2vec2-base-960h** (CTC head, 29-class character
15
+ vocabulary) with an **fp16-survivable log-probability tail**, for word-level forced
16
+ alignment. Consumed by the `alignkit` crate of the
17
+ [coremlit](https://github.com/findit-studio/coremlit) workspace.
18
+
19
+ ## Why this conversion exists
20
+
21
+ The stock conversion's tail is a decomposed `softmax → log(eps = 2^-149)`. That epsilon is
22
+ below fp16's smallest subnormal (`2^-24`), so on the Apple Neural Engine the guard rounds
23
+ to zero and `log(0)` saturates to **≈ −45440**, corrupting **16.7% of emission cells** and
24
+ shifting word timings by up to **881 ms** — silently, since the values are still finite and
25
+ negative. (A naive `torch.log_softmax` re-export reproduces the defect under coremltools
26
+ 9.0.) This conversion uses an explicitly **fused `x − logsumexp(x)`** tail: the emissions
27
+ are exact log-probabilities on CPU and bounded (min > −100) on every placement.
28
+
29
+ ## Contents
30
+
31
+ - `base960h_aligner.{mlpackage,mlmodelc}` — encoder weights **byte-identical** to the
32
+ established conversion (`weight.bin` sha256 `de51193f…`); only the tail graph changed.
33
+ - `CHECKSUMS.sha256` — per-file digests.
34
+
35
+ Input: 16 kHz mono f32, up to a 960,000-sample (60 s) window. Output: `[1, T, 29]`
36
+ log-probabilities, 320-sample hop.
37
+
38
+ ## Placement guidance (measured)
39
+
40
+ - **CPU: exact** log-probabilities (per-frame `logsumexp ≈ 0`) and the fastest placement
41
+ at this model size — the recommended default.
42
+ - **GPU / ANE: numerically safe with this conversion** (emissions bounded, no saturation),
43
+ but the wav2vec2 encoder itself is fp16-sensitive (group-norm): expect ~98% frame-argmax
44
+ agreement vs fp32, i.e. small word-timing differences. Consumers that require exact
45
+ timings should stay on CPU; the crate's runtime guard rejects any corrupted emission
46
+ matrix regardless of placement.
47
+
48
+ ## Verification (summary)
49
+
50
+ MIL-level fp16-guard audit clean (27 guard sites, every effective floor ≥ `2^-24`);
51
+ emissions min > −100 on ANE (−23.81) and GPU (−28.36) with real speech; word-timing parity
52
+ vs an independent ONNX implementation: median **0.0 ms** on unpadded 60 s audio (367/372
53
+ boundaries within one 20 ms frame). Toolchain pinned: coremltools 9.0, torch/torchaudio
54
+ 2.5.1, python 3.11.15.
55
+
56
+ ## Upstream provenance & licensing
57
+
58
+ Weights: [facebook/wav2vec2-base-960h](https://huggingface.co/facebook/wav2vec2-base-960h)
59
+ (Apache-2.0), via torchaudio's `WAV2VEC2_ASR_BASE_960H` pipeline. This is a derivative
60
+ conversion: identical weights, corrected graph numerics.
base960h_aligner.mlmodelc/analytics/coremldata.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d07ca6d8c69f2921810ca9bc919f0334f78de40a8f0c50e125358765fc5769fd
3
+ size 243
base960h_aligner.mlmodelc/coremldata.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:64b5214b50e003435d6659b8a3e4178e58ba48f3738daca0d6a1101412cb5a55
3
+ size 534
base960h_aligner.mlmodelc/metadata.json ADDED
@@ -0,0 +1,81 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "metadataOutputVersion" : "3.0",
4
+ "shortDescription" : "wav2vec2-base-960h forced aligner, epsilon-safe (reduce_log_sum_exp->sub tail)",
5
+ "outputSchema" : [
6
+ {
7
+ "hasShapeFlexibility" : "0",
8
+ "isOptional" : "0",
9
+ "dataType" : "Float32",
10
+ "formattedType" : "MultiArray (Float32 1 × 2999 × 29)",
11
+ "shortDescription" : "",
12
+ "shape" : "[1, 2999, 29]",
13
+ "name" : "emissions",
14
+ "type" : "MultiArray"
15
+ }
16
+ ],
17
+ "storagePrecision" : "Float16",
18
+ "modelParameters" : [
19
+
20
+ ],
21
+ "author" : "Phase-A reconversion: wav2vec2-base-960h aligner, stable fused log-softmax",
22
+ "specificationVersion" : 8,
23
+ "mlProgramOperationTypeHistogram" : {
24
+ "Ios17.mul" : 13,
25
+ "Ios17.sqrt" : 1,
26
+ "Ios17.square" : 1,
27
+ "Ios17.transpose" : 51,
28
+ "Ios17.sub" : 2,
29
+ "Ios17.linear" : 74,
30
+ "Ios17.conv" : 8,
31
+ "Ios17.matmul" : 24,
32
+ "Ios16.reduceLogSumExp" : 1,
33
+ "Ios17.add" : 27,
34
+ "Ios17.realDiv" : 1,
35
+ "Ios16.reduceMean" : 2,
36
+ "Ios17.sliceByIndex" : 1,
37
+ "Ios16.softmax" : 12,
38
+ "Ios16.gelu" : 20,
39
+ "Ios17.expandDims" : 1,
40
+ "Ios17.reshape" : 50,
41
+ "Ios17.cast" : 2,
42
+ "Ios17.layerNorm" : 26
43
+ },
44
+ "computePrecision" : "Mixed (Float16, Float32, Int32)",
45
+ "isUpdatable" : "0",
46
+ "stateSchema" : [
47
+
48
+ ],
49
+ "availability" : {
50
+ "macOS" : "14.0",
51
+ "tvOS" : "17.0",
52
+ "visionOS" : "1.0",
53
+ "watchOS" : "10.0",
54
+ "iOS" : "17.0",
55
+ "macCatalyst" : "17.0"
56
+ },
57
+ "modelType" : {
58
+ "name" : "MLModelType_mlProgram"
59
+ },
60
+ "inputSchema" : [
61
+ {
62
+ "hasShapeFlexibility" : "0",
63
+ "isOptional" : "0",
64
+ "dataType" : "Float32",
65
+ "formattedType" : "MultiArray (Float32 1 × 960000)",
66
+ "shortDescription" : "",
67
+ "shape" : "[1, 960000]",
68
+ "name" : "waveform",
69
+ "type" : "MultiArray"
70
+ }
71
+ ],
72
+ "userDefinedMetadata" : {
73
+ "com.github.apple.coremltools.conversion_date" : "2026-07-16",
74
+ "com.github.apple.coremltools.source" : "torch==2.5.1",
75
+ "com.github.apple.coremltools.version" : "9.0",
76
+ "com.github.apple.coremltools.source_dialect" : "TorchScript"
77
+ },
78
+ "generatedClassName" : "base960h_aligner",
79
+ "method" : "predict"
80
+ }
81
+ ]
base960h_aligner.mlmodelc/model.mil ADDED
The diff for this file is too large to render. See raw diff
 
base960h_aligner.mlmodelc/weights/weight.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:de51193fe73fb3aad085f9c794f08bfde1b939fc12f92e0834edcd4cb712e642
3
+ size 188799802
base960h_aligner.mlpackage/Data/com.apple.CoreML/model.mlmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ff4b7c3781d2a07698cb8883050697a45abb9e21325da3d9254df35139553c9c
3
+ size 154521
base960h_aligner.mlpackage/Data/com.apple.CoreML/weights/weight.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:de51193fe73fb3aad085f9c794f08bfde1b939fc12f92e0834edcd4cb712e642
3
+ size 188799802
base960h_aligner.mlpackage/Manifest.json ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "fileFormatVersion": "1.0.0",
3
+ "itemInfoEntries": {
4
+ "4296390D-EF20-44EA-B09C-2E9B874A1BA0": {
5
+ "author": "com.apple.CoreML",
6
+ "description": "CoreML Model Weights",
7
+ "name": "weights",
8
+ "path": "com.apple.CoreML/weights"
9
+ },
10
+ "EC9FDB7A-F224-4BE0-BE4A-1BD1C62458C4": {
11
+ "author": "com.apple.CoreML",
12
+ "description": "CoreML Model Specification",
13
+ "name": "model.mlmodel",
14
+ "path": "com.apple.CoreML/model.mlmodel"
15
+ }
16
+ },
17
+ "rootModelIdentifier": "EC9FDB7A-F224-4BE0-BE4A-1BD1C62458C4"
18
+ }