Fine-tuned on product search domain (brand, product name, origin)

README.md

@@ -1,5 +1,7 @@
 ---
 library_name: transformers
+license: cc-by-4.0
+base_model: bltlab/queryner-bert-base-uncased
 tags:
 - generated_from_trainer
 model-index:
@@ -12,9 +14,9 @@ should probably proofread and complete it, then remove this comment. -->
 
 # checkpoints
 
-This model was trained from scratch on the None dataset.
+This model is a fine-tuned version of [bltlab/queryner-bert-base-uncased](https://huggingface.co/bltlab/queryner-bert-base-uncased) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.1188
+- Loss: 0.2843
 
 ## Model description
 
@@ -46,8 +48,8 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 0.0812        | 1.0   | 1137 | 0.1426          |
-| 0.0569        | 2.0   | 2274 | 0.1188          |
+| 0.5758        | 1.0   | 1137 | 0.3628          |
+| 0.3382        | 2.0   | 2274 | 0.2843          |
 
 
 ### Framework versions

best/README.md

@@ -60,13 +60,13 @@ results = ner("organic olive oil from Italy under €15")
 
 ## Training data
 
-19,179 examples from three sources:
+20,203 examples from three sources:
 
 | Source | Examples | Notes |
 |---|---|---|
 | [bltlab/queryner](https://huggingface.co/datasets/bltlab/queryner) | 9,140 | Amazon ESCI queries; all 17 label types |
-| Local domain fixtures | ~1,000 | Hand-annotated product search queries |
-| Synthetic DB fixtures | ~9,000 | Template-generated from brand/category/product vocabulary |
+| Local domain fixtures | ~1,063 | Hand-annotated product search queries (incl. substitute-frame fixtures) |
+| Synthetic DB fixtures | ~10,000 | Template-generated from brand/category/product vocabulary; includes 1,000 substitute-frame (multilingual) |
 
 Synthetic examples are generated by `generate_db_dataset.py` from a European product database. Brand names come from EU-registered brands; product names are extracted from all language variants stored in `product.name` (en, de, fr, it, es, nl, and others). Product names that are exact matches of English category strings are excluded to avoid contradictory training signal.
 
@@ -136,21 +136,28 @@ Typical segment configuration:
 Segment 1: epochs=3, lr=3e-5   (base → domain)
 Segment 2: epochs=2, lr=1e-5   (add cert O-token signal)
 Segment 3: epochs=2, lr=5e-6   (product name ratio increase)
+Segment 4: epochs=2, lr=5e-6   (substitute-frame + multilingual, brand F1 0.698 → 0.897)
 ```
 
 ## Evaluation
 
-Evaluated on held-out domain fixtures with exact and partial span matching:
+Evaluated on 63 held-out domain fixtures (39 general + 24 substitute-frame / multilingual) with exact and partial span matching.
 
-| Label | Precision | Recall | F1 |
-|---|---|---|---|
-| brand | — | — | — |
-| product category | — | — | — |
-| product name | — | — | — |
-| origin | — | — | — |
-| **overall** | — | — | — |
+**Segment 4** — 2 epochs, lr=5e-6, base=segment 3 checkpoint, 20,203 training examples (incl. substitute-frame):
 
-*(Results updated after each training segment.)*
+| Label | P (partial) | R (partial) | F1 (partial) | F1 (exact) |
+|---|---|---|---|---|
+| brand | 0.929 | 0.867 | **0.897** | **0.897** |
+| product category | 0.895 | 0.962 | **0.927** | 0.891 |
+| product name | 0.875 | 0.700 | 0.778 | 0.556 |
+| origin | 1.000 | 0.917 | **0.957** | **0.957** |
+| **overall** | **0.915** | **0.900** | **0.908** | 0.874 |
+
+Key remaining gaps:
+- `Dr. Bronner's` apostrophe: tokenizer splits `'` → span predicted as `"dr. bronner ' s"`. Needs pre-tokenization normalization.
+- Ecover brand FN (4 fixtures): underrepresented in training vocabulary; missed even in substitute-frame context.
+- German origin `Deutschland` not recognized — training uses English country names only.
+- Umlaut span mismatch: `Spülmittel` lowercased to `spulmittel` by BERT WordPiece.
 
 ## Limitations
 

best/model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2ae80a09a730d7ed9c622d3941d055dc6aa3e78b4a8946027b1158df63646758
+oid sha256:4495433d8ccd4d56c332c6eee1286f22f51c85636ce0c748aac88fc9a10a2e6b
 size 435697596

best/training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c538014e65617630cb084588ec3ddf553c7fa06585fc03a0affc214c7993da69
+oid sha256:1d5f353af2b54f89774557b64e1037d093d33ff167ecdd7ab6dd86c911a4abad
 size 5969

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2ae80a09a730d7ed9c622d3941d055dc6aa3e78b4a8946027b1158df63646758
+oid sha256:4495433d8ccd4d56c332c6eee1286f22f51c85636ce0c748aac88fc9a10a2e6b
 size 435697596

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c538014e65617630cb084588ec3ddf553c7fa06585fc03a0affc214c7993da69
+oid sha256:1d5f353af2b54f89774557b64e1037d093d33ff167ecdd7ab6dd86c911a4abad
 size 5969