AutomatosX commited on
Commit
d1f30ce
·
verified ·
1 Parent(s): 1eb4ccd

Keep internal evaluation details private in model card

Browse files
Files changed (1) hide show
  1. README.md +20 -23
README.md CHANGED
@@ -120,40 +120,37 @@ one hardware/runtime configuration, not a latency guarantee.
120
  | Optimized ONNX CUDA FP32 | 34.423 | 40.770 | 27.76 | 76.2384 | 90.9896 | 86.8844 | 55.8988 |
121
  | Optimized ONNX CUDA + TF32 | 25.792 | 32.395 | 35.91 | 76.2414 | 90.9891 | 86.8792 | 55.9025 |
122
 
123
- AP uses all 11 classes on 6,489 development pages, with a 0.001 score floor
124
- for evaluation. The public prediction API uses a 0.30 score floor.
125
- Latency uses batch one,
126
- three counterbalanced passes of 100 pages after untimed startup samples.
127
- The timed region is decoded RGB to CPU box rows, including letterboxing,
128
- normalization, transfers, inference, validation and postprocessing; it excludes
129
- image-file decoding, IPC, initialization and evaluation. API serialization
130
- costs and host-load variation can change application latency. The measured p50
131
- reduction is 44.2% versus the eager reference. Small AP differences do not
132
- establish an accuracy improvement.
133
-
134
- On two additional financial-document development cohorts (256 and 100 pages),
135
- table AP changed by -0.0639 and +0.0730 points. The largest class AP decrease
136
- on the general development cohort was 0.0328 points. Corpus and per-page data
137
- are not included.
138
 
139
  The tested graph placed convolutions, matrix multiplications, GridSample and
140
  Softmax on CUDA. Some shape/control and arithmetic nodes execute on CPU;
141
  this is not an all-GPU graph claim.
142
 
143
- ## Release installation validation
144
 
145
- A separate clean installation of the CUDA-only inference package, with no
146
- PyTorch installed, was evaluated on all 6,489 general development pages:
147
 
148
  | AP | AP50 | AP70 | AP90 |
149
  |---:|---:|---:|---:|
150
  | 76.2396 | 90.9855 | 86.8767 | 55.9099 |
151
 
152
- The same installation passed the predefined development quality margins on
153
- both financial-document cohorts. An additional 128-page API check verified
154
- preprocessing and exact postprocessing of the captured inference outputs.
155
- Independent CUDA invocations are not bitwise-equivalent; these checks do not
156
- claim cross-session raw-detection equivalence or independent acceptance.
157
 
158
  ## Input and output
159
 
 
120
  | Optimized ONNX CUDA FP32 | 34.423 | 40.770 | 27.76 | 76.2384 | 90.9896 | 86.8844 | 55.8988 |
121
  | Optimized ONNX CUDA + TF32 | 25.792 | 32.395 | 35.91 | 76.2414 | 90.9891 | 86.8792 | 55.9025 |
122
 
123
+ AP averages IoU thresholds 0.50:0.95; AP50, AP70 and AP90 use their
124
+ respective IoU thresholds. Scores are percentages. Evaluation definitions
125
+ are private, so these numbers are not independently reproducible benchmarks.
126
+ The public prediction API uses a 0.30 score floor.
127
+
128
+ Latency measures batch-one decoded RGB to CPU box rows, including
129
+ preprocessing, transfers, inference, validation and postprocessing. It excludes
130
+ image decoding, IPC, initialization, evaluation and API serialization.
131
+ Application latency varies with host load. The measured median reduction is
132
+ 44.2% versus the eager reference. Small AP differences do not establish an
133
+ accuracy improvement.
134
+
135
+ Two additional internal evaluations showed AP changes of -0.0639 and +0.0730
136
+ points. The largest class AP decrease in the primary evaluation was 0.0328
137
+ points. Validate on representative inputs before deployment.
138
 
139
  The tested graph placed convolutions, matrix multiplications, GridSample and
140
  Softmax on CUDA. Some shape/control and arithmetic nodes execute on CPU;
141
  this is not an all-GPU graph claim.
142
 
143
+ ## Published inference runtime
144
 
145
+ The CUDA-only inference package does not require PyTorch. Its internal
146
+ development accuracy measurements are:
147
 
148
  | AP | AP50 | AP70 | AP90 |
149
  |---:|---:|---:|---:|
150
  | 76.2396 | 90.9855 | 86.8767 | 55.9099 |
151
 
152
+ These results are not independent acceptance. Independent CUDA invocations
153
+ are not bitwise equivalent.
 
 
 
154
 
155
  ## Input and output
156