Add files using upload-large-folder tool

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,267 @@
+---
+language:
+- en
+license: mit
+library_name: transformers
+pipeline_tag: text-classification
+base_model: BAAI/bge-m3
+inference: false
+metrics:
+- accuracy
+- recall
+tags:
+- hs-code
+- hs6
+- harmonized-system
+- hts
+- tariff
+- tariff-classification
+- customs
+- customs-clearance
+- trade-compliance
+- import-export
+- international-trade
+- logistics
+- supply-chain
+- ecommerce
+- product-classification
+- product-categorization
+- text-classification
+- multi-class-classification
+- english
+- xlm-roberta
+- bge-m3
+---
+
+# HS Code Classifier (HS6, English)
+
+**Give it an English product description, get back the 6-digit Harmonized System (HS)
+code** — the commodity code that drives customs tariff classification, duty rates, HTS
+lookups and trade compliance.
+
+```
+"men's cotton knitted t-shirt, short sleeve"           →  610910  (0.998)
+"portable laptop computer, 14 inch display, 1.2 kg"    →  847130  (0.994)
+"lithium-ion rechargeable battery, 3.7 V, 5000 mAh"    →  850760  (0.929)
+"roasted arabica coffee beans, not decaffeinated"      →  090121  (0.811)
+```
+
+Fine-tuned end-to-end over the full **6,750-class HS6 space** — not a toy subset of
+the most common chapters. HS4 (1,266 headings) and HS2 (97 chapters) come out of the
+same forward pass and are **guaranteed consistent** with the HS6 answer.
+
+| | |
+|---|---|
+| **Task** | Product description → HS6 tariff code |
+| **Input language** | **English** |
+| **Label space** | 6,750 HS6 · 1,266 HS4 · 97 HS2 |
+| **Accuracy on full product descriptions** | **71.94% top-1 · 91.08% top-5** |
+| **Parameters** | 573.6M (fp32, 2.2 GB) |
+| **Context window** | 1,024 tokens at inference |
+
+What you get depends on what you give it: a full description runs at ~72% top-1, a
+two-word catalogue stub at ~38%. The complete breakdown — by input type, by length, by
+confidence — is right below.
+
+---
+
+## Accuracy
+
+Measured on a frozen held-out test set of **8,065 product descriptions** that has zero
+overlap with training and was never used for tuning any hyperparameter or threshold.
+
+### Accuracy depends heavily on how much you tell it
+
+This is the single most important table in this card:
+
+| Input | Share of test | Top-1 | Top-5 |
+|---|---:|---:|---:|
+| **Full product description** | 35% | **71.94%** | **91.08%** |
+| Short subject line | 22% | 46.34% | 73.17% |
+| Two- or three-word phrase | 43% | 38.01% | 61.70% |
+
+By raw character length:
+
+| Length | Top-1 |
+|---|---:|
+| 0–40 chars | 39.00% |
+| 40–100 | 45.15% |
+| 100–200 | 46.98% |
+| 200–500 | 73.75% |
+| 500–1500 | **74.72%** |
+| 1500+ | 64.77% |
+
+**Do not truncate your inputs.** Material, construction, and intended use are exactly
+the features that separate one subheading from another; `"two folding cots"` does not
+contain them and no model can recover them. Feed the whole description — the spec
+sheet, the paragraph, the invoice line with attributes — and accuracy roughly doubles.
+
+### Averaged over the whole benchmark
+
+The benchmark is deliberately hostile: 62% of its inputs are under 100 characters, so
+the average is dragged down by fragments that carry no classifiable features at all.
+Averaged over all 8,065 items:
+
+| Metric | Score |
+|---|---|
+| HS6 top-1 (exact 6-digit match) | 51.70% |
+| HS4 top-1 (heading) | 60.99% |
+| HS2 top-1 (chapter) | 74.28% |
+| HS6 top-5 recall | 74.48% |
+
+Which of the two numbers applies to you is decided by your input, not by the model:
+feed full descriptions and expect the 72% row, feed catalogue stubs and expect the 38%
+row. Nothing is hidden here — both are stated so you can predict your own result
+before you download 2.2 GB.
+
+### Confidence is usable as a threshold
+
+Top-1 softmax probability is monotonically informative, which makes a
+straight-through / review split practical:
+
+| Confidence | Share of traffic | Accuracy |
+|---|---:|---:|
+| 0.8 – 1.0 | 62.8% | 63.7% |
+| 0.6 – 0.8 | 12.6% | 28.4% |
+| 0.4 – 0.6 | 12.3% | 21.6% |
+| 0.2 – 0.4 | 9.1% | 12.0% |
+| 0.0 – 0.2 | 3.2% | 3.1% |
+
+With temperature `T ≈ 1.75` applied to the logits, calibration improves further: you
+can auto-accept 41.5% of the flow at 80.0% accuracy. Recommended pattern — auto-clear
+high-confidence items, route the rest to a human with the top-5 list attached
+(top-5 covers 74% of everything and 91% of full descriptions).
+
+### Where the errors go
+
+| Outcome | Share |
+|---|---:|
+| Correct at top-1 | 51.70% |
+| Correct code present in top-5, but not ranked first | 22.78% |
+| Correct code absent from top-5 | 25.52% |
+
+Of the misses, 18.1% stay inside the correct HS4 heading and 26.6% inside the correct
+HS2 chapter — i.e. a large part of the error is near-miss, not nonsense.
+
+---
+
+## Quick start
+
+```bash
+pip install transformers torch sentencepiece
+```
+
+Verified on `transformers` 5.5 / `torch` 2.11. The model code uses only long-stable
+`transformers` APIs, so 4.4x and later should work as well.
+
+```python
+from transformers import AutoModel, AutoTokenizer
+
+REPO = "Kenpache/hs-code-classifier-en"
+model = AutoModel.from_pretrained(REPO, trust_remote_code=True).eval()
+tokenizer = AutoTokenizer.from_pretrained(REPO)
+
+model.classify(["men's cotton knitted t-shirt, short sleeve"], tokenizer, top_k=5)
+# [[{'hs6': '610910', 'score': 0.9983},
+#   {'hs6': '610990', 'score': 0.0010},
+#   {'hs6': '611020', 'score': 0.0005}, ...]]
+```
+
+`classify()` batches for you and accepts a list of any length:
+
+```python
+codes = model.classify(descriptions, tokenizer, top_k=5, batch_size=32)
+```
+
+### On GPU
+
+```python
+model = AutoModel.from_pretrained(REPO, trust_remote_code=True).to("cuda").eval()
+```
+
+CUDA, Apple Silicon (`mps`) and CPU all work. Roughly 3 GB of VRAM at batch 16 /
+length 1024; about 4 GB of RAM on CPU.
+
+**Leave the context window at 1,024 tokens.** It is already the default in
+`config.json`; lowering `recommended_max_length` costs about 2.9 points on texts longer
+than 1,500 characters, and raising it to 2,048 adds nothing.
+
+### All three HS levels at once
+
+```python
+import torch
+
+enc = tokenizer(["woven cotton fabric, dyed, 200 g/m2"], truncation=True,
+                max_length=1024, return_tensors="pt")
+with torch.no_grad():
+    out = model(**enc)
+
+hs6 = model.config.id2label[out.logits.argmax(-1).item()]     # '520839'
+hs4 = model.config.id2hs4[out.logits_hs4.argmax(-1).item()]   # '5208'
+hs2 = model.config.id2hs2[out.logits_hs2.argmax(-1).item()]   # '52'
+```
+
+`hs4` and `hs2` are marginals of the same distribution (logsumexp over the children
+of each parent), so **the levels can never contradict each other**: the model cannot
+return heading `6109` and a subheading that lives under `6110`.
+
+---
+
+## Limitations
+
+1. **English only.** The encoder is multilingual, but the head was trained on English
+   product descriptions. Other languages are untested and expected to be much weaker.
+2. **HS6 only.** The first six digits are internationally harmonized; national
+   8–10 digit tariff lines are out of scope and this model does not predict them.
+3. **Short inputs are hard**, as the tables above show. Under ~40 characters, expect
+   ~39% top-1.
+4. **Coverage is uneven across the 6,750 classes.** Rarely-seen codes are much weaker
+   than the headline number suggests; roughly 3,800 codes carry the bulk of the
+   model's competence.
+5. **The ceiling is domain-imposed, not model-imposed.** Identical descriptions
+   legitimately receive different codes depending on context (end use, material
+   composition, degree of processing), which caps any text-only classifier.
+6. **Not legal or customs advice.** Output is a ranked suggestion. Binding
+   classification is a decision of the competent authority. Use this to triage, to
+   pre-fill, and to route to a human — not to file unattended.
+
+---
+
+## Intended use
+
+Good fits:
+
+- pre-filling HS6 on customs declarations, then human review of low-confidence rows
+- catalogue / marketplace enrichment at scale
+- landed-cost and duty estimation tooling
+- deduplicating and sanity-checking existing classifications (flag rows where the
+  model is confident and disagrees)
+
+Poor fits: unattended filing, national tariff lines beyond 6 digits, non-English input.
+
+---
+
+## Files
+
+| File | What it is |
+|---|---|
+| `model.safetensors` | weights, fp32, 2.2 GB |
+| `config.json` | encoder config + head config + `id2label` for all 6,750 HS6 codes |
+| `modeling_hs6.py`, `configuration_hs6.py` | model definition (loaded via `trust_remote_code=True`) |
+| `tokenizer.json`, `sentencepiece.bpe.model`, … | XLM-R tokenizer, `model_max_length` 1024 |
+
+## License
+
+MIT, following the `BAAI/bge-m3` base model.
+
+## Citation
+
+```bibtex
+@misc{hs6_classifier_en,
+  title  = {HS Code Classifier (HS6, English)},
+  year   = {2026},
+  note   = {XLM-RoBERTa-large (bge-m3) with a flat 6,750-class HS6 head
+            and marginalized HS4/HS2 levels},
+  url    = {https://huggingface.co/Kenpache/hs-code-classifier-en}
+}
+```

configuration_hs6.py

@@ -0,0 +1,37 @@
+"""Configuration for the HS6 product classifier.
+
+Subclasses XLMRobertaConfig, so every encoder field (hidden_size, num_hidden_layers,
+...) keeps its usual name and meaning. Only the classification head and the label
+spaces for the coarser HS levels are added on top.
+"""
+
+from transformers.models.xlm_roberta.configuration_xlm_roberta import XLMRobertaConfig
+
+
+class HS6ClassifierConfig(XLMRobertaConfig):
+    model_type = "hs6_classifier"
+
+    def __init__(
+        self,
+        n2: int = 97,
+        n4: int = 1266,
+        n6: int = 6750,
+        head_dropout: float = 0.15,
+        pooling: str = "cls",
+        recommended_max_length: int = 1024,
+        id2hs4=None,
+        id2hs2=None,
+        **kwargs,
+    ):
+        super().__init__(**kwargs)
+        self.n2 = n2
+        self.n4 = n4
+        self.n6 = n6
+        self.head_dropout = head_dropout
+        self.pooling = pooling
+        # Training used 512 tokens; inference at 1024 is free (the encoder has 8194
+        # positions) and worth +2.91 points on texts longer than 1500 characters.
+        self.recommended_max_length = recommended_max_length
+        # HS4 / HS2 code strings, indexed by class id. Used to name the marginal logits.
+        self.id2hs4 = id2hs4 or []
+        self.id2hs2 = id2hs2 or []

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f15e074dc16b5869c4a855fb32e891fde314158af9ea916eb69fbc64d1ba8202
+size 2294652304

modeling_hs6.py

@@ -0,0 +1,127 @@
+"""HS6 product classifier: XLM-RoBERTa encoder + one flat linear head.
+
+HS4 and HS2 are not separate heads. They are marginals of the same HS6
+distribution (logsumexp over the children of each parent), so the levels are
+consistent by construction: the model cannot name one heading at 4 digits and a
+code from a different heading at 6 digits.
+
+The forward pass below must stay identical to the one used in training,
+otherwise the released weights do not mean what the metrics say they mean.
+"""
+
+from dataclasses import dataclass
+from typing import List, Optional
+
+import torch
+import torch.nn as nn
+from transformers.modeling_outputs import ModelOutput
+from transformers.modeling_utils import PreTrainedModel
+from transformers.models.xlm_roberta.configuration_xlm_roberta import XLMRobertaConfig
+from transformers.models.xlm_roberta.modeling_xlm_roberta import XLMRobertaModel
+
+from .configuration_hs6 import HS6ClassifierConfig
+
+# fields that belong to the classifier, not to the encoder
+_HEAD_ONLY = ("n2", "n4", "n6", "head_dropout", "pooling", "recommended_max_length",
+              "id2hs4", "id2hs2", "id2label", "label2id", "auto_map", "architectures",
+              "model_type")
+
+
+def _encoder_config(config: HS6ClassifierConfig) -> XLMRobertaConfig:
+    """A plain XLM-R config, so the encoder does not warn about the wrapper type."""
+    raw = {k: v for k, v in config.to_dict().items() if k not in _HEAD_ONLY}
+    return XLMRobertaConfig(**raw)
+
+
+@dataclass
+class HS6ClassifierOutput(ModelOutput):
+    """`logits` is the HS6 level, so the standard text-classification tooling works.
+
+    `logits_hs4` / `logits_hs2` are the marginals over the same distribution.
+    """
+
+    loss: Optional[torch.FloatTensor] = None
+    logits: Optional[torch.FloatTensor] = None
+    logits_hs4: Optional[torch.FloatTensor] = None
+    logits_hs2: Optional[torch.FloatTensor] = None
+
+
+class HS6ClassifierModel(PreTrainedModel):
+    config_class = HS6ClassifierConfig
+    base_model_prefix = "encoder"
+    supports_gradient_checkpointing = True
+
+    def __init__(self, config: HS6ClassifierConfig):
+        super().__init__(config)
+        # no pooling layer: the head reads the CLS token of the last hidden state
+        self.encoder = XLMRobertaModel(_encoder_config(config), add_pooling_layer=False)
+        self.pooling = config.pooling
+        self.drop = nn.Dropout(config.head_dropout)
+        self.head_6 = nn.Linear(config.hidden_size, config.n6)
+        self.n4, self.n2 = config.n4, config.n2
+        # parent of every HS6 class: its first 4 and first 2 digits. Stored in the
+        # checkpoint so the mapping cannot drift away from the trained weights.
+        self.register_buffer("parent4", torch.zeros(config.n6, dtype=torch.long))
+        self.register_buffer("parent2", torch.zeros(config.n6, dtype=torch.long))
+        self.post_init()
+
+    def _marginal(self, l6, parent, n_parent):
+        mx = l6.max(1, keepdim=True).values
+        e = (l6 - mx).exp()
+        s = torch.zeros(l6.size(0), n_parent, device=l6.device, dtype=e.dtype)
+        s.index_add_(1, parent, e)
+        return s.clamp_min(1e-20).log() + mx
+
+    def forward(
+        self,
+        input_ids: Optional[torch.LongTensor] = None,
+        attention_mask: Optional[torch.Tensor] = None,
+        labels: Optional[torch.LongTensor] = None,
+        return_dict: Optional[bool] = None,
+        **kwargs,
+    ):
+        return_dict = True if return_dict is None else return_dict
+        out = self.encoder(input_ids=input_ids, attention_mask=attention_mask)
+
+        if self.pooling == "mean":
+            h = out.last_hidden_state
+            m = attention_mask.unsqueeze(-1).to(h.dtype)
+            pooled = (h * m).sum(1) / m.sum(1).clamp(min=1)
+        else:
+            pooled = out.last_hidden_state[:, 0, :]
+
+        l6 = self.head_6(self.drop(pooled))
+        l4 = self._marginal(l6, self.parent4, self.n4)
+        l2 = self._marginal(l6, self.parent2, self.n2)
+
+        loss = None
+        if labels is not None:
+            loss = nn.functional.cross_entropy(l6, labels)
+
+        if not return_dict:
+            return (loss, l6, l4, l2) if loss is not None else (l6, l4, l2)
+        return HS6ClassifierOutput(loss=loss, logits=l6, logits_hs4=l4, logits_hs2=l2)
+
+    @torch.no_grad()
+    def classify(self, texts, tokenizer, top_k: int = 5, batch_size: int = 16,
+                 max_length: Optional[int] = None) -> List[List[dict]]:
+        """Convenience wrapper: texts in, ranked HS6 codes with probabilities out."""
+        if isinstance(texts, str):
+            texts = [texts]
+        max_length = max_length or self.config.recommended_max_length
+        device = next(self.parameters()).device
+        results = []
+        for start in range(0, len(texts), batch_size):
+            chunk = [t if isinstance(t, str) and t.strip() else " "
+                     for t in texts[start:start + batch_size]]
+            enc = tokenizer(chunk, truncation=True, max_length=max_length,
+                            padding=True, return_tensors="pt").to(device)
+            logits = self(**enc).logits
+            probs = torch.softmax(logits.float(), dim=1)
+            conf, idx = probs.topk(min(top_k, probs.size(1)), dim=1)
+            for c, i in zip(conf.cpu().tolist(), idx.cpu().tolist()):
+                results.append([
+                    {"hs6": self.config.id2label[j], "score": round(v, 6)}
+                    for j, v in zip(i, c)
+                ])
+        return results

sentencepiece.bpe.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cfc8146abe2a0488e9e2a0c56de7952f7c11ab059eca145a0a727afce0db2865
+size 5069051

special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5df1f55d60c9705a501ab9a75550728625740741fe4be308dac4806c16b7d51d
+size 17098085

tokenizer_config.json

@@ -0,0 +1,14 @@
+{
+  "add_prefix_space": true,
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "mask_token": "<mask>",
+  "model_max_length": 1024,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "sp_model_kwargs": {},
+  "tokenizer_class": "XLMRobertaTokenizer",
+  "unk_token": "<unk>"
+}