|
Download README.md from ruintheory/hts-classifier: direct link, hf CLI and curl.
- Browser
- Download file 8.36 kB
-
https://huggingface.co/ruintheory/hts-classifier/resolve/main/README.md
- Command line
-
hf download hf://ruintheory/hts-classifier/README.md
-
curl -L -o README.md https://huggingface.co/ruintheory/hts-classifier/resolve/main/README.md
8.36 kB
| license: apache-2.0 | |
| base_model: jinaai/jina-clip-v1 | |
| library_name: onnxruntime | |
| tags: | |
| - hts | |
| - harmonized-system | |
| - customs | |
| - tariff-classification | |
| - retrieval | |
| - onnx | |
| language: | |
| - en | |
| pipeline_tag: zero-shot-image-classification | |
| # HTS classifier (image and/or text → US HTS code + customs description) | |
| A CPU serving bundle for a demo. It maps a product photo and/or a product title to a 10-digit | |
| US Harmonized Tariff Schedule (HTS) code and returns a ≤30-character customs description, alternates, | |
| a confidence score, an abstain flag, and the nearest past customs rulings as justification. | |
| It runs on onnxruntime + `tokenizers` + numpy (no torch; the reranker's trees are evaluated in numpy). | |
| On 4 CPU threads: under 1 GB RSS, about 30–45 ms per text query including reranking, and about 250 ms with a photo. | |
| Bundle version: **r4 + reranker**. | |
| **Not legal advice.** Classification depends on facts a photo or title may not show | |
| (material, function, composition). Use the output as a starting point and confirm it against the | |
| HTS and with a licensed broker. | |
| ## How it works | |
| It retrieves and reranks rather than using a softmax classifier: | |
| 1. **Text tower**: the jina-clip-v1 text encoder, fully fine-tuned (768-d, mean pooling, | |
| queries up to 256 tokens). The loss is InfoNCE with a learned temperature, pulling queries | |
| toward their HTS leaf description and toward past rulings with the same code. | |
| Related codes (shared prefix) are masked out as false negatives. A small heading-level | |
| term and an anchor to the original model are added. Training ran in two phases on 566,982 | |
| query → code pairs, batch 256: 1,500 steps with tariff-tree sibling codes as hard negatives, | |
| then 2,500 steps with negatives mined by the phase-1 model. | |
| It is exported to ONNX with weight-only 8-bit `MatMulNBits` (block 128). | |
| 2. **Image tower**: the frozen jina-clip-v1 vision encoder (weight-only 8-bit ONNX) plus a | |
| 768×768 linear adapter with a fixed modality-gap shift. The adapter is trained to map photos | |
| into the fine-tuned text space. Photo + title queries use `norm(0.2·image + 0.8·text)`. | |
| 3. **Index**: 20,418 HTS leaf descriptions + 203,714 past ruling snippets, PCA 768→256, | |
| per-dimension int8, exact (brute-force) scoring with an int8 GEMM inside onnxruntime. | |
| 4. **Fusion**: leaf similarity and votes from the 100 nearest rulings (summed over 4/6/8/10-digit | |
| prefixes) are mixed 50/50 into a candidate shortlist. | |
| 5. **Reranker**: LightGBM (`lambdarank`, 250 trees × 7 leaves, monotone constraints) rescores | |
| the top 30 candidates with 24 features: fused rank and score, leaf gap, per-level ruling | |
| votes, keyword (BM25) match against the leaf text and plain-language aliases, "Other"-bucket | |
| and sibling counts, and stated-fact checks (gender, fiber, knit/woven, quantities). It was trained only on 2023 validation rulings plus half of an extra CBP set, | |
| using 5-fold out-of-fold training grouped by gold code. Benchmarks and 2024+ test rulings were never in its training data. | |
| 6. **Confidence**: the reranker's softmax probability mass on the top 6-digit code. The demo | |
| reports the digits it is confident in by walking down the tariff tree while one branch holds | |
| ≥ 0.6 of the mass. Below that depth it asks a clarifying question. | |
| ## Files | |
| | file | what | | |
| |---|---| | |
| | `text.onnx` | fine-tuned text tower, weight-only 8-bit (input `input_ids`, batch 1, ≤256 tokens) | | |
| | `vision.onnx` | jina-clip-v1 vision tower, weight-only 8-bit (input `pixel_values` [1,3,224,224]) | | |
| | `tokenizer.json` | tokenizer for `text.onnx` | | |
| | `image_adapter.npz` | `W`, `b`, `gap` of the image adapter | | |
| | `index.npz` | PCA matrix (768→256) and per-dim int8 scales | | |
| | `score_leaf.onnx`, `score_pool.onnx` | int8 GEMM scorers holding the leaf / ruling embeddings | | |
| | `leaves.parquet` | HTS leaf codes, descriptions, paths (HTS revision downloaded 2026-11-12) | | |
| | `pool.parquet` | ruling snippets: `label` (code), `source`, `text` | | |
| | `rerank.json` | reranker trees, feature list, softmax temperature (T = 2), abstain threshold | | |
| | `rr_aliases.json` | plain-language aliases per leaf, used by the reranker's keyword features | | |
| | `meta.json` | serving settings (`max_tokens`) | | |
| Serving code: `hts/serve.py` and `hts/engine.py` in the project repository. | |
| ## Evaluation (shipped path: fused shortlist → reranker, strict hit rule) | |
| A prediction counts as correct only if it matches the gold code to the stated depth. Every query | |
| is in the denominator, so a ruling whose gold code is shorter than 10 digits counts as a | |
| 10-digit miss. Training uses US rulings dated 2022 or earlier; validation uses 2023; the test sets use 2024 or later. | |
| Near-duplicate rulings are merged before scoring. Intervals are 95% Wilson intervals. | |
| | set | n | R@10 | top-1 HS6 | top-1 10-digit | | |
| |---|---|---|---|---| | |
| | US CBP rulings 2024+, full description | 2,855 | 0.890 | 0.658 | 0.415 | | |
| | US CBP rulings 2024+, short description | 2,480 | 0.787 | 0.533 | 0.304 | | |
| | HSCodeComp (product titles, text only) | 632 | 0.752 | 0.503 [0.464, 0.542] | 0.342 [0.306, 0.380] | | |
| | ATLAS test (product titles) | 200 | 0.710 | 0.540 [0.471, 0.608] | 0.335 [0.273, 0.403] | | |
| What the reranker adds over the fused shortlist on the same bundle: +0.6 to +2.5 points top-1 | |
| 10-digit on the test sets (−0.3 on out-of-fold validation), +2 to +5 points R@10, and fewer | |
| clarifying questions. On 500 real listings, questions per answer dropped from 1.04 to 0.73. | |
| Real marketplace listings (500 product pages; gold = the depth at which two independent LLM | |
| labelers agreed, usually 4–6 digits; this set was also used for model selection): 0.784 top-1. | |
| Photos: | |
| - ABO product photos (725 items, 29 product types, heading level): image only 0.67 top-1 heading, | |
| photo + title 0.89 through the reranker. ABO was also used to tune photo weights. | |
| - 424 real listings with photos: photo + title 0.792 vs title only 0.767 | |
| (+13/−2 items, sign test p = .007). | |
| **Abstain threshold:** abstain when the top HS6 mass is below 0.375. On out-of-fold validation | |
| the model answers 56% of queries, with 74% HS6 precision on those answers. This is a probability mass, | |
| not a calibrated probability of being correct. Calibration has not been checked on the test | |
| sets or on photo inputs. | |
| All results are from a single training seed. Quantization was validated end to end on an earlier | |
| model (within about 1 point; top-1 agreement with fp32 0.99). The numbers above are measured | |
| through the shipped 8-bit stack. | |
| ## Data and attribution | |
| Ruling snippets in `pool.parquet` and the training data come from: | |
| - US CBP CROSS rulings (US government work), via the `orlandowhite/hts-cross-rulings` | |
| and `fklc/cbp-rulings-past-2012` (MIT) datasets. | |
| - EU Binding Tariff Information (EBTI) and EU classification regulations, © European | |
| Union, reused under Commission Decision 2011/833/EU / CC BY 4.0. | |
| - UK Advance Tariff Rulings, © Crown copyright, Open Government Licence v3.0. | |
| - Australian Tariff Concession Orders, © Commonwealth of Australia (Department of Home | |
| Affairs), CC BY 3.0 AU. | |
| - Canada Border Services Agency (CBSA) advance rulings: 1,201 snippets (0.6% of the pool). | |
| - US HTS from the US International Trade Commission. | |
| - About half of the training rows are real ruling text. The rest are LLM-generated or | |
| LLM-labeled: synthetic shopper-style titles (Qwen3-8B), plain-language aliases per leaf | |
| (Claude Sonnet), and public fashion catalog listings (Myntra, H&M) with LLM-voted labels. | |
| These were used for training only, except for the aliases in `rr_aliases.json`. | |
| - The image adapter was trained on embeddings of product photos from Amazon Reviews 2023 | |
| (Hou et al., McAuley Lab, UCSD) and Open Images crops. No images or review text are | |
| included in this repository. | |
| Base model: [jinaai/jina-clip-v1](https://huggingface.co/jinaai/jina-clip-v1) (Apache-2.0). | |
| ## Limitations | |
| - US HTS only, with codes from the HTS revision above. Chapters 98 and 99 are excluded. | |
| - Product titles that lack material or function details (e.g. "insulated bottle") can land in | |
| the wrong heading. The abstain flag catches many of these, but not all. | |
| - Image-only queries are reliable at heading level and much less so at 10 digits. | |
| - English only (non-English input is not translated at serve time). | |
| - Accuracy drops sharply for codes with few past rulings. | |