House style: replace em dashes

32f68aa verified 5 days ago

4.72 kB

license: cc-by-nc-4.0
pipeline_tag: image-classification
library_name: onnx
tags:
  - onnx
  - toy-reference
  - prompt2model
  - automl
  - image-classification
extra_gated_heading: Acknowledge the terms to access this model
extra_gated_prompt: >-
  Access is granted for non-commercial research and evaluation use only. By
  requesting access you agree not to redistribute this model or any derivative
  artifacts, to cite Dhi Technologies in any publication or output that uses it,
  and to obtain a separate commercial license via dhi-tech.com before any
  commercial use. Access requests are reviewed manually by Dhi Technologies.
extra_gated_fields:
  Name: text
  Affiliation: text
  Intended use: text
  I agree to use this model for non-commercial research and evaluation only, and not to redistribute it: checkbox
extra_gated_button_content: Submit request

prompt2model-reference-onnx

This is a toy reference artifact, not a production model. It is the ONNX classification export that the public prompt2model-demo Space's conformal-abstain browser evaluates. Its purpose is to give people a real, small, fast-loading model file to poke at while looking at how Prompt2Model's factory pipeline (prompt to config to training to metrics to ONNX export to report) behaves end to end, not to demonstrate classification accuracy on any task that matters.

What it is

Task: image classification, 3 synthetic classes: red_square, blue_circle, green_triangle.
Backbone: mobilenet_v3_small, ImageNet-pretrained (the smoke test path hardcodes a pretrained backbone as of PR #18).
Parameters: 1,520,931 (about 1.52M).
Input: 96x96 RGB images.
Export format: ONNX, opset 17.
Training data: a tiny synthetic toy set (colored shapes), 2 epochs, batch size 8, low_light augmentation tag, generated by the same run that produced this export. Not a natural-image dataset. See prompt2model-examples for the matching example fixtures.

Measured metrics (verbatim, this run only)

This is the deterministic smoke run after PR #18 (seeded RNGs, stratified splits, pretrained backbone), the same artifact the demo Space evaluates.

{
  "accuracy": 1.0,
  "macro_f1": 1.0,
  "latency_ms": 78.61008350737393,
  "fps": 12.721014345522176,
  "parameter_count": 1520931,
  "calibration": {
    "calibrated": true,
    "temperature": 0.05,
    "alpha": 0.1,
    "conformal_threshold": 0.004888,
    "ece_before": 0.3864,
    "ece_after": 0.0014,
    "val_samples": 7
  }
}

Chance level on this 3-class set is 0.33. Accuracy 1.0 / macro F1 1.0 is expected and not impressive: the toy set is tiny (a handful of synthetic images per class), the task is trivially separable (three fixed colored shapes), and the split is small enough that a perfect score carries very little statistical weight. Treat these numbers as "the pipeline plumbing works end to end," not "this model classifies well."
Latency/FPS were measured locally on a development machine (Apple Silicon Mac, ONNX Runtime CPU execution provider) via the same onnxruntime.InferenceSession the model ships in, not a production or edge-device benchmark.
Calibration numbers are from the pipeline's conformal-abstain stage (temperature scaling + conformal threshold), which is what the demo Space's "abstain browser" actually visualizes.

What this is not

Not a production or deployment-ready classifier.
Not evidence of real-world classification quality: the label set, images, and split are all synthetic and tiny.
Not tuned, ablated, or compared against any baseline beyond the trivial chance rate.

Provenance

Repo: DHI-Technologies-Inc/Prompt2Model-Language-Guided-Vision-Model-Factory
Commit: ba62c2fd721115f996812af61079ad339a8c5b80 ("fix(training): seed RNGs, stratify splits, use pretrained backbone in smoke test (#18)")
Command: .venv/bin/python -m prompt2model.cli smoke-test --output-dir output/smoke
Date: 2026-07-10

License and access

Released under cc-by-nc-4.0 and gated for non-commercial research and evaluation only. No redistribution. Commercial licensing via dhi-tech.com.

Dhi-Technologies
/

prompt2model-reference-onnx