Flor commited on
Commit
b96f6c0
·
verified ·
1 Parent(s): 0fff7e5

Model card synced with fastforks 7204e33f

Browse files
Files changed (1) hide show
  1. README.md +12 -4
README.md CHANGED
@@ -50,13 +50,19 @@ Tested up to (portable, fp16, RTX 5060 Ti 16 GB): `efficientnet_v2_s.tv_in1k` 38
50
 
51
  | model | time (ms): original fp32 | original | portable | kernels | kernels speed-up: vs original | vs portable | peak memory (MiB, allocated / reserved): original | portable | kernels |
52
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
53
- | `efficientnet_v2_s.tv_in1k` | 12.14 | 16.6 | 10.18 | 2.11 | 5.8x | 4.8x | 159 / 176 | 81 / 106 | 113 / 168 |
54
 
55
  ### Startup and first call
56
 
57
  | model | load (s): original | portable | kernels | first call (s): original | portable | kernels |
58
  | --- | --- | --- | --- | --- | --- | --- |
59
- | `efficientnet_v2_s.tv_in1k` | 1.48 | 0.35 | 0.53† | 0.97 | 0.04 | 0.01 |
 
 
 
 
 
 
60
 
61
  ### Fixed-size speed and peak memory, all models
62
 
@@ -73,14 +79,14 @@ Hardware, protocol and caveats: [benchmarks](https://github.com/kleinicke/fastfo
73
 
74
  ### Parity with the original code
75
 
76
- Every variant is checked against golden outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 0.002 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.02 px and p99 <= 0.08 px.
77
 
78
  ## Usage
79
 
80
  ```python
81
  from fastforks import classification
82
 
83
- model = classification.load("efficientnet_v2_s.tv_in1k") # portable PyTorch on the GPU if there is one, else the CPU
84
  result = model("photo.jpg")
85
  result.top(5) # top-5 classes
86
  result.backend # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16')
@@ -91,3 +97,5 @@ Install with `pip install fastforks` (see <https://github.com/kleinicke/fastfork
91
  ## Kernels backend
92
 
93
  `classification.load("efficientnet_v2_s.tv_in1k", mode="kernels")` runs our CUDA kernels (`cudnn_hacc`, `fvm_conv`, `fvm_extra`, `fvm_mb`, `fvm_r3`). It needs a prebuilt kernel for your GPU architecture (downloaded from <https://huggingface.co/fastforks/kernels> on first use) and a recent NVIDIA driver; if there is none it raises `BackendUnavailableError` with the reason and never falls back silently. `mode="auto"` picks `kernels` when it can run and logs the choice.
 
 
 
50
 
51
  | model | time (ms): original fp32 | original | portable | kernels | kernels speed-up: vs original | vs portable | peak memory (MiB, allocated / reserved): original | portable | kernels |
52
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
53
+ | `efficientnet_v2_s.tv_in1k` | 12.14 | 16.6 | 10.18 | 1.893 | 6.4x | 5.4x | 159 / 176 | 81 / 106 | 114 / 188 |
54
 
55
  ### Startup and first call
56
 
57
  | model | load (s): original | portable | kernels | first call (s): original | portable | kernels |
58
  | --- | --- | --- | --- | --- | --- | --- |
59
+ | `efficientnet_v2_s.tv_in1k` | 1.48 | 0.35 | 1.90 | 0.97 | 0.04 | 0.05 |
60
+
61
+ ### torch.compile and TensorRT
62
+
63
+ | model | input | portable ms | compile ms | compile s (cold) | TensorRT fp16 ms | ONNX + build s | kernels ms | peak MiB (alloc / reserved): portable | compile | TensorRT (device) |
64
+ | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
65
+ | `efficientnet_v2_s.tv_in1k` | 224x224 | 10.18 | 5.4 | 12.1 | 1.0 | 28 | 1.893 | 81 / 106 | 116 / 146 | 68 |
66
 
67
  ### Fixed-size speed and peak memory, all models
68
 
 
79
 
80
  ### Parity with the original code
81
 
82
+ Every variant is checked against golden outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 0.002 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.02 and p99 <= 0.08 (absolute error of the output `logits`).
83
 
84
  ## Usage
85
 
86
  ```python
87
  from fastforks import classification
88
 
89
+ model = classification.load("efficientnet_v2_s.tv_in1k") # mode="auto": the preferred backend from recorded measurements
90
  result = model("photo.jpg")
91
  result.top(5) # top-5 classes
92
  result.backend # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16')
 
97
  ## Kernels backend
98
 
99
  `classification.load("efficientnet_v2_s.tv_in1k", mode="kernels")` runs our CUDA kernels (`cudnn_hacc`, `fvm_conv`, `fvm_extra`, `fvm_mb`, `fvm_r3`). It needs a prebuilt kernel for your GPU architecture (downloaded from <https://huggingface.co/fastforks/kernels> on first use) and a recent NVIDIA driver; if there is none it raises `BackendUnavailableError` with the reason and never falls back silently. `mode="auto"` picks `kernels` when it can run and logs the choice.
100
+
101
+ <!-- staged from fastforks 7204e33f (2026-11-05) -->