justinchuby commited on
Commit
9ecbf77
·
verified ·
1 Parent(s): 1efbb9a

Create a model without the DFT node

Browse files

The shaves another 5ms in the onnx model.

```
python scripts/compare_onnx_tflite.py --onnx perch_v2_no_dft
.onnx --tflite perch_v2.tflite --benchmark --atol 2e-4 --rtol 2e-4
Loading ONNX model from: perch_v2_no_dft.onnx
Loading TFLite model from: perch_v2.tflite
/opt/homebrew/Caskroom/miniconda/base/envs/tf2onnx/lib/python3.12/site-packages/tensorflow/lite/python/interpreter.py:457: UserWarning: Warning: tf.lite.Interpreter is deprecated and is scheduled for deletion in
TF 2.20. Please use the LiteRT interpreter from the ai_edge_litert package.
See the [migration guide](https://ai.google.dev/edge/litert/migration)
for details.

warnings.warn(_INTERPRETER_DELETION_WARNING)
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.

ONNX Model Information:
Inputs:
- Name: inputs, Shape: ['batch', 160000], Type: tensor(float)
Outputs:
- Name: embedding, Shape: ['batch', 1536], Type: tensor(float)
- Name: spatial_embedding, Shape: ['batch', 16, 4, 1536], Type: tensor(float)
- Name: spectrogram, Shape: ['batch', 500, 128], Type: tensor(float)
- Name: label, Shape: ['batch', 14795], Type: tensor(float)

TFLite Model Information:
Inputs:
- Name: serving_default_inputs:0, Shape: [ 1 160000], Type: <class 'numpy.float32'>
Outputs:
- Name: StatefulPartitionedCall:0, Shape: [ 1 1536], Type: <class 'numpy.float32'>
- Name: StatefulPartitionedCall:2, Shape: [ 1 16 4 1536], Type: <class 'numpy.float32'>
- Name: StatefulPartitionedCall:3, Shape: [ 1 500 128], Type: <class 'numpy.float32'>
- Name: StatefulPartitionedCall:1, Shape: [ 1 14795], Type: <class 'numpy.float32'>

Generating random inputs:
- inputs: shape=(1, 160000), dtype=float32

Running ONNX model inference...
Running TFLite model inference...

================================================================================
COMPARISON RESULTS
================================================================================

Output 0:
ONNX Runtime shape: (1, 1536), dtype: float32
TFLite shape: (1, 1536), dtype: float32

ONNX Runtime vs TFLite:
Max difference: 0.0000016540
Mean difference: 0.0000003347
Relative tolerance: 0.0002
Absolute tolerance: 0.0002
✅ Outputs match within tolerance

Output 1:
ONNX Runtime shape: (1, 16, 4, 1536), dtype: float32
TFLite shape: (1, 16, 4, 1536), dtype: float32

ONNX Runtime vs TFLite:
Max difference: 0.0000883937
Mean difference: 0.0000016283
Relative tolerance: 0.0002
Absolute tolerance: 0.0002
✅ Outputs match within tolerance

Output 2:
ONNX Runtime shape: (1, 500, 128), dtype: float32
TFLite shape: (1, 500, 128), dtype: float32

ONNX Runtime vs TFLite:
Max difference: 0.0000153184
Mean difference: 0.0000001984
Relative tolerance: 0.0002
Absolute tolerance: 0.0002
✅ Outputs match within tolerance

Output 3:
ONNX Runtime shape: (1, 14795), dtype: float32
TFLite shape: (1, 14795), dtype: float32

ONNX Runtime vs TFLite:
Max difference: 0.0001187325
Mean difference: 0.0000043795
Relative tolerance: 0.0002
Absolute tolerance: 0.0002
✅ Outputs match within tolerance

================================================================================
✅ ALL OUTPUTS MATCH!
================================================================================

Benchmarking ONNX model (10 warmup + 100 test runs)...
Benchmarking TFLite model (10 warmup + 100 test runs)...

================================================================================
BENCHMARK RESULTS
================================================================================

ONNX Model:
Mean: 58.609 ms
Median: 58.104 ms
Std: 1.811 ms
Min: 55.502 ms
Max: 63.099 ms

TFLite Model:
Mean: 612.875 ms
Median: 606.164 ms
Std: 22.839 ms
Min: 601.629 ms
Max: 754.459 ms

Comparison:
ONNX Runtime is 10.46x faster than TFLite
Difference: 554.265 ms
================================================================================
```

Files changed (1) hide show
  1. perch_v2_no_dft.onnx +3 -0
perch_v2_no_dft.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4dcf71c18a147198545944bb5149697e89e3ad2e16637fa8f0edf6d13035a017
3
+ size 413350933