Hugging Face's logo

Unseen1980
/
mockgen-classifier-v1

Text Classification
ONNX
onnxruntime
mock-data
semantic-classification
enterprise-schemas
Model card Files
xet
 Community
mockgen-classifier-v1 / README.md
Unseen1980's picture
Unseen1980
Update README.md
17024ee verified
preview code
|
Raw
History Blame Contribute Delete
7.78 kB
metadata 
license: apache-2.0
base_model: mockgen-word-sgd-v1
library_name: onnxruntime
pipeline_tag: text-classification
tags:
  - mock-data
  - semantic-classification
  - enterprise-schemas
widget:
  - text: |-
      Entity: JournalEntryItem
      Field: SAKNR
      Label: G/L Account
      Type: string
      Related: CompanyCode, FiscalYear, AmountInCompanyCodeCurrency
    example_title: SAP G/L account field
    output:
      - label: gl_account
        score: 0.6323356628417969
      - label: country
        score: 0.02091727964580059
      - label: fiscal_period
        score: 0.012878832407295704
  - text: |-
      Entity: AccountingDocument
      Field: BUKRS
      Label: Company Code
      Type: string
      Related: CompanyCodeName, Country, Currency, ChartOfAccounts
    example_title: Company code field
    output:
      - label: company_code
        score: 0.6074062585830688
      - label: currency
        score: 0.1046433299779892
      - label: country
        score: 0.07778121531009674
  - text: |-
      Entity: SupplierInvoice
      Field: ZTERM
      Label: Terms of Payment
      Type: string
      Related: Supplier, InvoiceAmount, DueDate, PaymentBlockingReason
    example_title: Payment terms field
    output:
      - label: payment_terms
        score: 0.7714565992355347
      - label: country
        score: 0.023687832057476044
      - label: currency
        score: 0.010493538342416286
  - text: |-
      Entity: MaterialDocumentItem
      Field: WERKS
      Label: Plant
      Type: string
      Related: StorageLocation, Material, QuantityInEntryUnit
    example_title: Plant field
    output:
      - label: plant
        score: 0.5506921410560608
      - label: material_no
        score: 0.036951154470443726
      - label: country
        score: 0.036014873534440994
  - text: |-
      Entity: PurchaseOrderItem
      Field: MSEHI
      Label: Order Unit
      Type: string
      Related: Material, OrderQuantity, NetPriceAmount
    example_title: Order unit field
    output:
      - label: unit_of_measure
        score: 0.7284214496612549
      - label: quantity
        score: 0.07117477804422379
      - label: purchase_order_no
        score: 0.03281358629465103
  - text: |-
      Entity: SalesOrder
      Field: KUNNR
      Label: Sold-To Party
      Type: string
      Related: SalesDocument, SalesOrganization, CustomerName, NetAmount
    example_title: Sold-to party field
    output:
      - label: customer_id
        score: 0.4585545063018799
      - label: org_name
        score: 0.05491480976343155
      - label: country
        score: 0.030577752739191055
  - text: |-
      Entity: SalesOrder
      Field: CurrencyCode
      Label: Currency
      Type: string
      Related: NetAmount, GrossAmount, Country
    example_title: Currency field
    output:
      - label: currency
        score: 0.46692952513694763
      - label: country
        score: 0.3333224654197693
      - label: org_name
        score: 0.019802909344434738

MockGen Classifier v1

Artifact status

The classifier is trained locally, export verified, and published to Hugging Face. This is the accepted v1 classifier artifact. Experimental retraining runs are evaluated offline and only replace this artifact when they improve both service-holdout and domain-holdout benchmarks.

Publication status

Public repository: https://huggingface.co/Unseen1980/mockgen-classifier-v1/tree/v1

Published artifacts are the quantized ONNX classifier, this card, and non-secret metrics. The publication-safe classifier dataset is published separately at https://huggingface.co/datasets/Unseen1980/mockgen-classifier-data-v1/tree/v1.

The card includes pipeline_tag: text-classification and widget examples so the Hub can display the classifier's task and sample outputs. Live browser inference depends on Hugging Face provider support for this repository. For a dedicated Hugging Face Inference Endpoint, publish the included handler.py and requirements.txt; the handler wraps the ONNX model with the same field-text normalization used by MockGen.

Intended use

The classifier predicts MockGen semantic vocabulary hints for enterprise schema properties. The runtime uses those hints to route properties to retrieval banks, deterministic patterns, grounded templates, and validated fallbacks.

Local inference

The classifier can be tested without a Fiori app:

npx tsx -e '
import { readFileSync } from "node:fs";
import { createOnnxClassifierRunner } from "./packages/mockgen-core/src/classifier-runner";

(async () => {
  const report = JSON.parse(readFileSync("models/classifier/onnx/export-report.json", "utf8"));
  const runner = await createOnnxClassifierRunner({
    modelPath: "packages/mockgen-models/artifacts/classifier.onnx",
    hints: report.labels,
    inputNormalizer: report.inputNormalizer
  });

  const result = await runner.classify({
    entity: "JournalEntryItem",
    name: "SAKNR",
    type: "string",
    constraints: { nullable: true },
    semantic: {
      hint: "unknown",
      label: "G/L Account",
      related: ["CompanyCode", "FiscalYear", "AmountInCompanyCodeCurrency"]
    },
    values: [],
    examples: []
  });

  console.log(JSON.stringify({
    hint: result.hint,
    confidence: result.confidence,
    top: result.top.slice(0, 5)
  }, null, 2));
})();
'

Expected top prediction:

{
  "hint": "gl_account",
  "confidence": 0.6323356628417969
}

The raw ONNX input is a single string tensor named text. MockGen normally builds this string from property metadata, for example:

property_name: SAKNR [SEP] label: G/L Account [SEP] entity_type_name: JournalEntryItem [SEP] neighbor_properties: CompanyCode, FiscalYear, AmountInCompanyCodeCurrency [SEP] type: string

Training data

Training data came from reviewed schema-property rows built during the cross-family classifier pipeline. The public dataset repository contains the publication-safe subset: project-authored fixture rows, reviewed public API field metadata, and reviewed recovery fixtures with source/provenance metadata. The exportable checkpoint uses a word-sgd backend with unicode-codepoint-v1 normalization rather than a pretrained encoder, because this path produced a compact ONNX artifact with stable multilingual signal.

Evaluation

Local validation evidence:

  • Training rows: 21,533; validation rows: 2,393.
  • Full-precision validation top-1: 0.9820309235269536.
  • Full-precision validation top-3: 0.9887170915169243.
  • Quantized ONNX validation top-1: 0.9816130380275805.
  • Quantized ONNX validation top-3: 0.9887170915169243.
  • Absolute top-1 drop after quantization: 0.00041788549937316066, below the 0.01 gate.
  • Quantized ONNX size: 5.101428985595703 MiB, below the 50 MiB gate.
  • Quantized ONNX SHA-256: b8827b79fc4def92ab6dbac7990a708771a8e72693df63c55f3b4f88d420ded9.
  • Classifier replacement gate: a new trained artifact must improve both the service-holdout macro-F1 baseline 0.8247 and the domain-holdout macro-F1 baseline 0.7073.
  • Most recent experimental retraining status: not published because it did not improve both benchmarks.

Limitations

The v1 vocabulary has about 50 trained hints with reserved slots for expansion. Expected weaknesses include rare enterprise fields, ambiguous labels without neighboring-property context, non-Western naming conventions, and schema domains not represented in the benchmark corpus.

Reproducibility

Use models/classifier/training-config.yaml, models/classifier/metrics.json, models/classifier/onnx/export-report.json, and the paired dataset repository as the local reproducibility anchors. Public artifact checksums are verified from the model repository after publication.