Create a model without the DFT node
Browse filesThe shaves another 5ms in the onnx model.
```
python scripts/compare_onnx_tflite.py --onnx perch_v2_no_dft
.onnx --tflite perch_v2.tflite --benchmark --atol 2e-4 --rtol 2e-4
Loading ONNX model from: perch_v2_no_dft.onnx
Loading TFLite model from: perch_v2.tflite
/opt/homebrew/Caskroom/miniconda/base/envs/tf2onnx/lib/python3.12/site-packages/tensorflow/lite/python/interpreter.py:457: UserWarning: Warning: tf.lite.Interpreter is deprecated and is scheduled for deletion in
TF 2.20. Please use the LiteRT interpreter from the ai_edge_litert package.
See the [migration guide](https://ai.google.dev/edge/litert/migration)
for details.
warnings.warn(_INTERPRETER_DELETION_WARNING)
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
ONNX Model Information:
Inputs:
- Name: inputs, Shape: ['batch', 160000], Type: tensor(float)
Outputs:
- Name: embedding, Shape: ['batch', 1536], Type: tensor(float)
- Name: spatial_embedding, Shape: ['batch', 16, 4, 1536], Type: tensor(float)
- Name: spectrogram, Shape: ['batch', 500, 128], Type: tensor(float)
- Name: label, Shape: ['batch', 14795], Type: tensor(float)
TFLite Model Information:
Inputs:
- Name: serving_default_inputs:0, Shape: [ 1 160000], Type: <class 'numpy.float32'>
Outputs:
- Name: StatefulPartitionedCall:0, Shape: [ 1 1536], Type: <class 'numpy.float32'>
- Name: StatefulPartitionedCall:2, Shape: [ 1 16 4 1536], Type: <class 'numpy.float32'>
- Name: StatefulPartitionedCall:3, Shape: [ 1 500 128], Type: <class 'numpy.float32'>
- Name: StatefulPartitionedCall:1, Shape: [ 1 14795], Type: <class 'numpy.float32'>
Generating random inputs:
- inputs: shape=(1, 160000), dtype=float32
Running ONNX model inference...
Running TFLite model inference...
================================================================================
COMPARISON RESULTS
================================================================================
Output 0:
ONNX Runtime shape: (1, 1536), dtype: float32
TFLite shape: (1, 1536), dtype: float32
ONNX Runtime vs TFLite:
Max difference: 0.0000016540
Mean difference: 0.0000003347
Relative tolerance: 0.0002
Absolute tolerance: 0.0002
✅ Outputs match within tolerance
Output 1:
ONNX Runtime shape: (1, 16, 4, 1536), dtype: float32
TFLite shape: (1, 16, 4, 1536), dtype: float32
ONNX Runtime vs TFLite:
Max difference: 0.0000883937
Mean difference: 0.0000016283
Relative tolerance: 0.0002
Absolute tolerance: 0.0002
✅ Outputs match within tolerance
Output 2:
ONNX Runtime shape: (1, 500, 128), dtype: float32
TFLite shape: (1, 500, 128), dtype: float32
ONNX Runtime vs TFLite:
Max difference: 0.0000153184
Mean difference: 0.0000001984
Relative tolerance: 0.0002
Absolute tolerance: 0.0002
✅ Outputs match within tolerance
Output 3:
ONNX Runtime shape: (1, 14795), dtype: float32
TFLite shape: (1, 14795), dtype: float32
ONNX Runtime vs TFLite:
Max difference: 0.0001187325
Mean difference: 0.0000043795
Relative tolerance: 0.0002
Absolute tolerance: 0.0002
✅ Outputs match within tolerance
================================================================================
✅ ALL OUTPUTS MATCH!
================================================================================
Benchmarking ONNX model (10 warmup + 100 test runs)...
Benchmarking TFLite model (10 warmup + 100 test runs)...
================================================================================
BENCHMARK RESULTS
================================================================================
ONNX Model:
Mean: 58.609 ms
Median: 58.104 ms
Std: 1.811 ms
Min: 55.502 ms
Max: 63.099 ms
TFLite Model:
Mean: 612.875 ms
Median: 606.164 ms
Std: 22.839 ms
Min: 601.629 ms
Max: 754.459 ms
Comparison:
ONNX Runtime is 10.46x faster than TFLite
Difference: 554.265 ms
================================================================================
```
- perch_v2_no_dft.onnx +3 -0
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4dcf71c18a147198545944bb5149697e89e3ad2e16637fa8f0edf6d13035a017
|
| 3 |
+
size 413350933
|