Upload mf-0.1

README.md

@@ -0,0 +1,132 @@
+---
+license: mit
+base_model: BAAI/bge-small-en-v1.5
+library_name: transformers
+pipeline_tag: text-classification
+language: en
+tags:
+- construction
+- masterformat
+- text-classification
+- spec-writing
+- takeoff
+- bert
+- onnx
+metrics:
+- accuracy
+model-index:
+- name: masterformat-classifier
+  results:
+  - task:
+      type: text-classification
+      name: MasterFormat level-2 classification
+    dataset:
+      type: ufgs-val
+      name: UFGS held-out text units (13,518)
+      split: validation
+    metrics:
+    - type: accuracy
+      value: 0.184
+      name: Top-1 accuracy (step-700 checkpoint)
+---
+
+# MasterFormat level-2 classifier (mf-0.1)
+
+> **Early checkpoint, published as-is.** This is step 700 of a planned 4,150-step fine-tune; training was
+> paused on an 8 GB CPU-only Mac. Top-1 accuracy on the full UFGS validation split is **18.4%** — well below the
+> TF-IDF baseline (53.1%) on the same split. Treat this release as a reproducible starting point and a
+> working deployment format, not as a finished classifier. A resume script and the full recipe are in the
+> source project.
+
+Classifies a construction line item, specification paragraph, submittal or section title into one of **171
+MasterFormat level-2 groups** (e.g. `03 30 00 Cast-in-Place Concrete`, `23 30 00 HVAC Air Distribution`)
+across 32 divisions. English text, one label per input.
+
+MasterFormat is a registered trademark of CSI / CSC. This project is not affiliated with or endorsed by them.
+
+## Model
+
+- Base: [`BAAI/bge-small-en-v1.5`](https://huggingface.co/BAAI/bge-small-en-v1.5) (BERT, 33M parameters, 384-d), MIT.
+- Fine-tuned with a 171-way sequence-classification head; `id2label` / `label2id` are in [`config.json`](config.json).
+- Training: 2 epochs planned over 66,396 rows (per-class balance capped at 450 / floored at 300, estimate-style
+  augmentation), batch 32, max length 96, AdamW, 6% linear warmup; body LR 6e-5, head LR 2e-3; embeddings and
+  the first 4 encoder layers frozen.
+- Checkpoint stopped at step 700; `train_state.json` records `{"step": 700, "val_acc": 0.18725}`.
+
+## Results
+
+All numbers are measured on held-out data from the same project. `val` = UFGS text units,
+`line_items` = 194 hand-labelled estimate line items, `manuals` = 6,890 chunks from 153 sections of two real
+commercial project manuals (out-of-taxonomy gold labels count only toward the division score).
+
+| Scorer | val top-1 | val top-3 | val division | line-item top-1 | manual-chunk top-1 | manual-section top-1 |
+|---|---:|---:|---:|---:|---:|---:|
+| **mf-0.1 (step 700)** | 0.184 | 0.345 | 0.321 | 0.283 | 0.175 | 0.287 |
+| TF-IDF + linear SGD (full run) | 0.531 | 0.690 | 0.634 | 0.665 | 0.290 | 0.607 |
+| bge-small embeddings + logistic regression | 0.349 | 0.517 | 0.458 | 0.576 | 0.256 | 0.593 |
+| Ensemble (0.25 embedding + 0.75 TF-IDF) | 0.532 | 0.700 | 0.637 | 0.702 | 0.318 | 0.640 |
+
+## Quick start
+
+```python
+from transformers import pipeline
+
+clf = pipeline("text-classification", model="constructelligence/masterformat-classifier", top_k=3)
+for p in clf("4000 psi concrete slab on grade"):
+    print(p["label"], round(p["score"], 3))
+```
+
+Or with the included example:
+
+```bash
+pip install transformers torch
+python predict.py "TPO roofing, 60 mil, fully adhered"
+```
+
+The encoder is English-only; inputs should be English.
+
+### ONNX
+
+`onnx/model.onnx` (fp32, 134 MB) and `onnx/model_quantized.onnx` (dynamic int8, 34 MB) are exported with
+dynamic batch and sequence axes, in the layout `transformers.js` / Optimum expect, inputs `input_ids`,
+`attention_mask`, `token_type_ids`, output `logits`. The fp32 export matches PyTorch on a 4-text smoke set
+(max |delta logit| 5e-6, argmax agreement 4/4). Dynamic int8 quantization shifts low-confidence logits
+(max |delta| ~1.4 on the same smoke set, argmax agreement 3/4), so verify the quantized model on your own
+inputs before using it; see the project's `scripts/export_onnx.py` for the exact command.
+
+## Files
+
+- `model.safetensors`, `config.json`, `tokenizer.json`, `tokenizer_config.json`, `vocab.txt`,
+  `special_tokens_map.json` — standard `transformers` checkpoint.
+- `train_state.json` — step and validation accuracy of the saved checkpoint (text logs and optimizer state
+  are not published).
+- `predict.py` — minimal pipeline example.
+- `onnx/` — ONNX fp32 and int8 exports.
+
+## Training data and taxonomy
+
+- **UFGS** (Unified Facilities Guide Specifications) `.SEC` files, US federal works in the **public domain**.
+  Paragraphs and titles are parsed into labelled text units; boilerplate shared by 3+ sections is dropped,
+  cross-references are stripped so section numbers cannot leak labels, and long paragraphs are windowed into
+  20–60-word units.
+- **Split:** held out by a hash of the normalised source text (10%), so a paragraph and its augmentations stay
+  on the same side. 66,396 train / 13,518 val rows.
+- **Taxonomy:** 171 level-2 groups over 32 MasterFormat divisions; group numbers follow the MasterFormat
+  numbering convention and the short names are the project's own (see `config.json`).
+
+## Limitations
+
+- **Early checkpoint.** The 18.4% top-1 is a snapshot of an interrupted run; it is not competitive with the
+  TF-IDF baseline yet. Do not use it for production takeoff without further training and re-evaluation.
+- **Domain skew.** UFGS over-represents heavy-civil, water/wastewater and process work relative to commercial
+  building estimates; the training set is class-balanced, so raw predictions do not reflect building-project
+  priors.
+- **Out-of-taxonomy inputs** (sections whose level-2 group is not in the 171) can only be scored at division
+  level.
+- Short, terse line items are the hardest inputs; division (2-digit) accuracy is higher than group accuracy.
+
+## Licence and attribution
+
+Released under the **MIT licence**, matching the base model. UFGS source text is public domain. MasterFormat
+is a registered trademark of CSI / CSC; this project is not affiliated with or endorsed by them. Constructelligence's
+production models are available at [constructelligence.co](https://constructelligence.co).

config.json

@@ -0,0 +1,371 @@
+{
+  "architectures": [
+    "BertForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "dtype": "float32",
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 384,
+  "id2label": {
+    "0": "01 10 00 Summary of Work",
+    "1": "01 20 00 Price and Payment Procedures",
+    "2": "01 30 00 Administrative Requirements",
+    "3": "01 40 00 Quality Requirements",
+    "4": "01 50 00 Temporary Facilities and Controls",
+    "5": "01 70 00 Execution and Closeout Requirements",
+    "6": "01 90 00 Commissioning and Life Cycle Activities",
+    "7": "02 30 00 Subsurface Investigation",
+    "8": "02 40 00 Demolition and Structure Moving",
+    "9": "02 50 00 Site Remediation",
+    "10": "02 60 00 Contaminated Site Material Removal",
+    "11": "02 70 00 Water Remediation",
+    "12": "02 80 00 Facility Remediation (Asbestos, Lead, Mold)",
+    "13": "03 00 00 Concrete Rehabilitation and Common Work",
+    "14": "03 10 00 Concrete Forming and Accessories",
+    "15": "03 20 00 Concrete Reinforcing",
+    "16": "03 30 00 Cast-in-Place Concrete",
+    "17": "03 40 00 Precast Concrete",
+    "18": "03 50 00 Cast Decks and Underlayment",
+    "19": "03 60 00 Grouting",
+    "20": "03 70 00 Mass Concrete",
+    "21": "04 00 00 Masonry Restoration and Common Work",
+    "22": "04 20 00 Unit Masonry",
+    "23": "05 00 00 Metals Common Work (Anchors, Welding)",
+    "24": "05 10 00 Structural Metal Framing",
+    "25": "05 20 00 Metal Joists",
+    "26": "05 30 00 Metal Decking",
+    "27": "05 40 00 Cold-Formed Metal Framing",
+    "28": "05 50 00 Metal Fabrications",
+    "29": "05 70 00 Decorative Metal",
+    "30": "06 10 00 Rough Carpentry",
+    "31": "06 20 00 Finish Carpentry",
+    "32": "06 40 00 Architectural Woodwork",
+    "33": "06 60 00 Plastic Fabrications",
+    "34": "06 70 00 Structural Composites (FRP)",
+    "35": "06 80 00 Composite Fabrications",
+    "36": "07 10 00 Dampproofing and Waterproofing",
+    "37": "07 20 00 Thermal Protection (Insulation, Air Barriers)",
+    "38": "07 30 00 Steep Slope Roofing",
+    "39": "07 40 00 Roofing and Siding Panels",
+    "40": "07 50 00 Membrane Roofing",
+    "41": "07 60 00 Flashing and Sheet Metal",
+    "42": "07 70 00 Roof and Wall Specialties and Accessories",
+    "43": "07 80 00 Fire and Smoke Protection",
+    "44": "07 90 00 Joint Protection (Sealants)",
+    "45": "08 00 00 Openings Maintenance and Common Work",
+    "46": "08 10 00 Doors and Frames",
+    "47": "08 20 00 Composite and Plastic Doors",
+    "48": "08 30 00 Specialty Doors and Frames",
+    "49": "08 40 00 Entrances, Storefronts, and Curtain Walls",
+    "50": "08 50 00 Windows",
+    "51": "08 60 00 Roof Windows and Skylights",
+    "52": "08 70 00 Door Hardware",
+    "53": "08 80 00 Glazing",
+    "54": "08 90 00 Louvers and Vents",
+    "55": "09 00 00 Finishes Preparation and Schedules",
+    "56": "09 20 00 Plaster and Gypsum Board",
+    "57": "09 30 00 Tiling",
+    "58": "09 50 00 Ceilings",
+    "59": "09 60 00 Flooring",
+    "60": "09 70 00 Wall Finishes",
+    "61": "09 80 00 Acoustic Treatment",
+    "62": "09 90 00 Painting and Coating",
+    "63": "10 10 00 Information Specialties (Signage, Boards)",
+    "64": "10 20 00 Interior Specialties (Partitions, Accessories)",
+    "65": "10 40 00 Safety Specialties",
+    "66": "10 50 00 Storage Specialties (Lockers, Shelving)",
+    "67": "11 00 00 Equipment Common Work",
+    "68": "11 10 00 Vehicle and Loading Dock Equipment",
+    "69": "11 30 00 Residential Equipment",
+    "70": "11 40 00 Foodservice Equipment",
+    "71": "11 50 00 Educational and Scientific Equipment",
+    "72": "11 60 00 Entertainment and Recreation Equipment",
+    "73": "11 70 00 Healthcare Equipment",
+    "74": "11 80 00 Collection and Disposal Equipment",
+    "75": "12 00 00 Furnishings General (FF&E)",
+    "76": "12 20 00 Window Treatments",
+    "77": "12 30 00 Casework",
+    "78": "12 40 00 Furnishings and Accessories",
+    "79": "12 50 00 Furniture",
+    "80": "12 60 00 Multiple Seating",
+    "81": "12 90 00 Other Furnishings (Site Furnishings)",
+    "82": "13 10 00 Special Facility Components",
+    "83": "13 20 00 Special Purpose Rooms",
+    "84": "13 30 00 Special Structures (Pre-Engineered Buildings)",
+    "85": "13 40 00 Integrated Construction (Shielding, Seismic)",
+    "86": "13 50 00 Special Instrumentation",
+    "87": "14 20 00 Elevators",
+    "88": "14 90 00 Other Conveying Equipment",
+    "89": "21 10 00 Water-Based Fire Suppression (Sprinklers, Standpipes)",
+    "90": "21 20 00 Fire-Extinguishing Systems",
+    "91": "21 30 00 Fire Pumps",
+    "92": "22 00 00 Plumbing General (incl. Fixtures)",
+    "93": "22 10 00 Plumbing Piping and Pumps",
+    "94": "22 30 00 Plumbing Equipment (Water Heaters, Softeners)",
+    "95": "22 60 00 Gas and Vacuum Systems for Healthcare and Labs",
+    "96": "23 00 00 HVAC Common Work (Piping, Insulation, TAB, Controls)",
+    "97": "23 10 00 Facility Fuel Systems",
+    "98": "23 20 00 HVAC Piping and Pumps",
+    "99": "23 30 00 HVAC Air Distribution",
+    "100": "23 40 00 HVAC Air Cleaning Devices",
+    "101": "23 50 00 Central Heating Equipment",
+    "102": "23 60 00 Central Cooling Equipment",
+    "103": "23 70 00 Central HVAC Equipment",
+    "104": "23 80 00 Decentralized HVAC Equipment",
+    "105": "25 00 00 Integrated Automation Common Work",
+    "106": "25 10 00 Integrated Automation Network Equipment",
+    "107": "26 00 00 Electrical Common Work (Wire, Cable, Grounding)",
+    "108": "26 10 00 Medium-Voltage Electrical Distribution",
+    "109": "26 20 00 Low-Voltage Electrical Distribution",
+    "110": "26 30 00 Power Generating and Storing Equipment",
+    "111": "26 40 00 Electrical and Cathodic Protection",
+    "112": "26 50 00 Lighting",
+    "113": "27 00 00 Communications Common Work",
+    "114": "27 10 00 Structured Cabling",
+    "115": "27 40 00 Audio-Video Communications",
+    "116": "27 50 00 Distributed Communications and Monitoring",
+    "117": "28 00 00 Electronic Safety and Security Common Work",
+    "118": "28 10 00 Electronic Security and Access Control",
+    "119": "28 20 00 Electronic Surveillance and Monitoring",
+    "120": "28 30 00 Detection and Alarm (incl. Fire Alarm)",
+    "121": "31 00 00 Earthwork General",
+    "122": "31 10 00 Site Clearing",
+    "123": "31 20 00 Earth Moving",
+    "124": "31 30 00 Earthwork Methods (Soil Treatment, Stabilization)",
+    "125": "31 40 00 Shoring and Underpinning",
+    "126": "31 50 00 Excavation Support and Protection",
+    "127": "31 60 00 Special Foundations (Piles, Caissons)",
+    "128": "31 70 00 Tunneling and Mining",
+    "129": "32 00 00 Exterior Improvements Maintenance",
+    "130": "32 10 00 Bases, Ballasts, and Paving",
+    "131": "32 30 00 Site Improvements (Fences, Gates)",
+    "132": "32 80 00 Irrigation",
+    "133": "32 90 00 Planting",
+    "134": "33 00 00 Utilities Maintenance and Common Work",
+    "135": "33 10 00 Water Utilities",
+    "136": "33 20 00 Wells",
+    "137": "33 30 00 Sanitary Sewerage Utilities",
+    "138": "33 40 00 Storm Drainage Utilities",
+    "139": "33 50 00 Fuel Distribution Utilities",
+    "140": "33 60 00 Hydronic and Steam Energy Utilities",
+    "141": "33 70 00 Electrical Utilities",
+    "142": "33 80 00 Communications Utilities",
+    "143": "34 10 00 Guideways and Railways",
+    "144": "34 60 00 Vehicle Charging Facilities",
+    "145": "34 70 00 Transportation Construction and Equipment",
+    "146": "35 00 00 Waterway and Marine Common Work",
+    "147": "35 20 00 Waterway and Marine Construction Equipment (Gates)",
+    "148": "35 30 00 Coastal Construction",
+    "149": "35 40 00 Waterway Construction (Levees, Pumps)",
+    "150": "35 50 00 Marine Construction (Piers, Fenders)",
+    "151": "35 70 00 Dam Construction",
+    "152": "40 00 00 Process Piping and Common Work",
+    "153": "40 10 00 Gas and Vapor Process Piping",
+    "154": "40 60 00 Process Control",
+    "155": "41 20 00 Piece Material Handling Equipment (Cranes, Hoists)",
+    "156": "41 30 00 Manufacturing Equipment",
+    "157": "41 60 00 Mobile Plant Equipment",
+    "158": "43 20 00 Liquid Handling Equipment",
+    "159": "43 30 00 Gas and Liquid Purification Equipment",
+    "160": "44 10 00 Air Pollution Control",
+    "161": "44 40 00 Water Pollution Control",
+    "162": "46 00 00 Water and Wastewater Equipment Common Work",
+    "163": "46 20 00 Preliminary Treatment Equipment",
+    "164": "46 30 00 Chemical Feed Equipment",
+    "165": "46 40 00 Clarification and Mixing Equipment",
+    "166": "46 50 00 Secondary Treatment Equipment",
+    "167": "46 60 00 Advanced Treatment and Disinfection Equipment",
+    "168": "46 70 00 Residuals Handling and Treatment",
+    "169": "48 00 00 Power Generation Common Work",
+    "170": "48 10 00 Power Generation Equipment (Solar, Wind)"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 1536,
+  "label2id": {
+    "01 10 00 Summary of Work": 0,
+    "01 20 00 Price and Payment Procedures": 1,
+    "01 30 00 Administrative Requirements": 2,
+    "01 40 00 Quality Requirements": 3,
+    "01 50 00 Temporary Facilities and Controls": 4,
+    "01 70 00 Execution and Closeout Requirements": 5,
+    "01 90 00 Commissioning and Life Cycle Activities": 6,
+    "02 30 00 Subsurface Investigation": 7,
+    "02 40 00 Demolition and Structure Moving": 8,
+    "02 50 00 Site Remediation": 9,
+    "02 60 00 Contaminated Site Material Removal": 10,
+    "02 70 00 Water Remediation": 11,
+    "02 80 00 Facility Remediation (Asbestos, Lead, Mold)": 12,
+    "03 00 00 Concrete Rehabilitation and Common Work": 13,
+    "03 10 00 Concrete Forming and Accessories": 14,
+    "03 20 00 Concrete Reinforcing": 15,
+    "03 30 00 Cast-in-Place Concrete": 16,
+    "03 40 00 Precast Concrete": 17,
+    "03 50 00 Cast Decks and Underlayment": 18,
+    "03 60 00 Grouting": 19,
+    "03 70 00 Mass Concrete": 20,
+    "04 00 00 Masonry Restoration and Common Work": 21,
+    "04 20 00 Unit Masonry": 22,
+    "05 00 00 Metals Common Work (Anchors, Welding)": 23,
+    "05 10 00 Structural Metal Framing": 24,
+    "05 20 00 Metal Joists": 25,
+    "05 30 00 Metal Decking": 26,
+    "05 40 00 Cold-Formed Metal Framing": 27,
+    "05 50 00 Metal Fabrications": 28,
+    "05 70 00 Decorative Metal": 29,
+    "06 10 00 Rough Carpentry": 30,
+    "06 20 00 Finish Carpentry": 31,
+    "06 40 00 Architectural Woodwork": 32,
+    "06 60 00 Plastic Fabrications": 33,
+    "06 70 00 Structural Composites (FRP)": 34,
+    "06 80 00 Composite Fabrications": 35,
+    "07 10 00 Dampproofing and Waterproofing": 36,
+    "07 20 00 Thermal Protection (Insulation, Air Barriers)": 37,
+    "07 30 00 Steep Slope Roofing": 38,
+    "07 40 00 Roofing and Siding Panels": 39,
+    "07 50 00 Membrane Roofing": 40,
+    "07 60 00 Flashing and Sheet Metal": 41,
+    "07 70 00 Roof and Wall Specialties and Accessories": 42,
+    "07 80 00 Fire and Smoke Protection": 43,
+    "07 90 00 Joint Protection (Sealants)": 44,
+    "08 00 00 Openings Maintenance and Common Work": 45,
+    "08 10 00 Doors and Frames": 46,
+    "08 20 00 Composite and Plastic Doors": 47,
+    "08 30 00 Specialty Doors and Frames": 48,
+    "08 40 00 Entrances, Storefronts, and Curtain Walls": 49,
+    "08 50 00 Windows": 50,
+    "08 60 00 Roof Windows and Skylights": 51,
+    "08 70 00 Door Hardware": 52,
+    "08 80 00 Glazing": 53,
+    "08 90 00 Louvers and Vents": 54,
+    "09 00 00 Finishes Preparation and Schedules": 55,
+    "09 20 00 Plaster and Gypsum Board": 56,
+    "09 30 00 Tiling": 57,
+    "09 50 00 Ceilings": 58,
+    "09 60 00 Flooring": 59,
+    "09 70 00 Wall Finishes": 60,
+    "09 80 00 Acoustic Treatment": 61,
+    "09 90 00 Painting and Coating": 62,
+    "10 10 00 Information Specialties (Signage, Boards)": 63,
+    "10 20 00 Interior Specialties (Partitions, Accessories)": 64,
+    "10 40 00 Safety Specialties": 65,
+    "10 50 00 Storage Specialties (Lockers, Shelving)": 66,
+    "11 00 00 Equipment Common Work": 67,
+    "11 10 00 Vehicle and Loading Dock Equipment": 68,
+    "11 30 00 Residential Equipment": 69,
+    "11 40 00 Foodservice Equipment": 70,
+    "11 50 00 Educational and Scientific Equipment": 71,
+    "11 60 00 Entertainment and Recreation Equipment": 72,
+    "11 70 00 Healthcare Equipment": 73,
+    "11 80 00 Collection and Disposal Equipment": 74,
+    "12 00 00 Furnishings General (FF&E)": 75,
+    "12 20 00 Window Treatments": 76,
+    "12 30 00 Casework": 77,
+    "12 40 00 Furnishings and Accessories": 78,
+    "12 50 00 Furniture": 79,
+    "12 60 00 Multiple Seating": 80,
+    "12 90 00 Other Furnishings (Site Furnishings)": 81,
+    "13 10 00 Special Facility Components": 82,
+    "13 20 00 Special Purpose Rooms": 83,
+    "13 30 00 Special Structures (Pre-Engineered Buildings)": 84,
+    "13 40 00 Integrated Construction (Shielding, Seismic)": 85,
+    "13 50 00 Special Instrumentation": 86,
+    "14 20 00 Elevators": 87,
+    "14 90 00 Other Conveying Equipment": 88,
+    "21 10 00 Water-Based Fire Suppression (Sprinklers, Standpipes)": 89,
+    "21 20 00 Fire-Extinguishing Systems": 90,
+    "21 30 00 Fire Pumps": 91,
+    "22 00 00 Plumbing General (incl. Fixtures)": 92,
+    "22 10 00 Plumbing Piping and Pumps": 93,
+    "22 30 00 Plumbing Equipment (Water Heaters, Softeners)": 94,
+    "22 60 00 Gas and Vacuum Systems for Healthcare and Labs": 95,
+    "23 00 00 HVAC Common Work (Piping, Insulation, TAB, Controls)": 96,
+    "23 10 00 Facility Fuel Systems": 97,
+    "23 20 00 HVAC Piping and Pumps": 98,
+    "23 30 00 HVAC Air Distribution": 99,
+    "23 40 00 HVAC Air Cleaning Devices": 100,
+    "23 50 00 Central Heating Equipment": 101,
+    "23 60 00 Central Cooling Equipment": 102,
+    "23 70 00 Central HVAC Equipment": 103,
+    "23 80 00 Decentralized HVAC Equipment": 104,
+    "25 00 00 Integrated Automation Common Work": 105,
+    "25 10 00 Integrated Automation Network Equipment": 106,
+    "26 00 00 Electrical Common Work (Wire, Cable, Grounding)": 107,
+    "26 10 00 Medium-Voltage Electrical Distribution": 108,
+    "26 20 00 Low-Voltage Electrical Distribution": 109,
+    "26 30 00 Power Generating and Storing Equipment": 110,
+    "26 40 00 Electrical and Cathodic Protection": 111,
+    "26 50 00 Lighting": 112,
+    "27 00 00 Communications Common Work": 113,
+    "27 10 00 Structured Cabling": 114,
+    "27 40 00 Audio-Video Communications": 115,
+    "27 50 00 Distributed Communications and Monitoring": 116,
+    "28 00 00 Electronic Safety and Security Common Work": 117,
+    "28 10 00 Electronic Security and Access Control": 118,
+    "28 20 00 Electronic Surveillance and Monitoring": 119,
+    "28 30 00 Detection and Alarm (incl. Fire Alarm)": 120,
+    "31 00 00 Earthwork General": 121,
+    "31 10 00 Site Clearing": 122,
+    "31 20 00 Earth Moving": 123,
+    "31 30 00 Earthwork Methods (Soil Treatment, Stabilization)": 124,
+    "31 40 00 Shoring and Underpinning": 125,
+    "31 50 00 Excavation Support and Protection": 126,
+    "31 60 00 Special Foundations (Piles, Caissons)": 127,
+    "31 70 00 Tunneling and Mining": 128,
+    "32 00 00 Exterior Improvements Maintenance": 129,
+    "32 10 00 Bases, Ballasts, and Paving": 130,
+    "32 30 00 Site Improvements (Fences, Gates)": 131,
+    "32 80 00 Irrigation": 132,
+    "32 90 00 Planting": 133,
+    "33 00 00 Utilities Maintenance and Common Work": 134,
+    "33 10 00 Water Utilities": 135,
+    "33 20 00 Wells": 136,
+    "33 30 00 Sanitary Sewerage Utilities": 137,
+    "33 40 00 Storm Drainage Utilities": 138,
+    "33 50 00 Fuel Distribution Utilities": 139,
+    "33 60 00 Hydronic and Steam Energy Utilities": 140,
+    "33 70 00 Electrical Utilities": 141,
+    "33 80 00 Communications Utilities": 142,
+    "34 10 00 Guideways and Railways": 143,
+    "34 60 00 Vehicle Charging Facilities": 144,
+    "34 70 00 Transportation Construction and Equipment": 145,
+    "35 00 00 Waterway and Marine Common Work": 146,
+    "35 20 00 Waterway and Marine Construction Equipment (Gates)": 147,
+    "35 30 00 Coastal Construction": 148,
+    "35 40 00 Waterway Construction (Levees, Pumps)": 149,
+    "35 50 00 Marine Construction (Piers, Fenders)": 150,
+    "35 70 00 Dam Construction": 151,
+    "40 00 00 Process Piping and Common Work": 152,
+    "40 10 00 Gas and Vapor Process Piping": 153,
+    "40 60 00 Process Control": 154,
+    "41 20 00 Piece Material Handling Equipment (Cranes, Hoists)": 155,
+    "41 30 00 Manufacturing Equipment": 156,
+    "41 60 00 Mobile Plant Equipment": 157,
+    "43 20 00 Liquid Handling Equipment": 158,
+    "43 30 00 Gas and Liquid Purification Equipment": 159,
+    "44 10 00 Air Pollution Control": 160,
+    "44 40 00 Water Pollution Control": 161,
+    "46 00 00 Water and Wastewater Equipment Common Work": 162,
+    "46 20 00 Preliminary Treatment Equipment": 163,
+    "46 30 00 Chemical Feed Equipment": 164,
+    "46 40 00 Clarification and Mixing Equipment": 165,
+    "46 50 00 Secondary Treatment Equipment": 166,
+    "46 60 00 Advanced Treatment and Disinfection Equipment": 167,
+    "46 70 00 Residuals Handling and Treatment": 168,
+    "48 00 00 Power Generation Common Work": 169,
+    "48 10 00 Power Generation Equipment (Solar, Wind)": 170
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "problem_type": "single_label_classification",
+  "transformers_version": "4.57.6",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0770e36bd9b8e8f054ac5750ab0da3078c1e9b4d00ce907a90e9b9f59fa048ca
+size 133726636

onnx/model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c359f7e6387049e671c1c2039c9e65003f80205183a5702134ebf0b4a39e3fe8
+size 133960852

onnx/model_quantized.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3096cf891670cf74be258a21c860825dd3e4bd3aa3f4b48bb5e6d716ff4fca5a
+size 34067631

predict.py

@@ -0,0 +1,25 @@
+"""Classify construction text into MasterFormat level-2 groups.
+
+    pip install transformers torch
+    python predict.py "4000 psi concrete slab on grade" "TPO roofing, 60 mil, fully adhered"
+
+Without arguments it classifies one example line item.
+"""
+import sys
+
+from transformers import pipeline
+
+MODEL = "constructelligence/masterformat-classifier"
+
+
+def main():
+    texts = sys.argv[1:] or ["4000 psi concrete slab on grade"]
+    clf = pipeline("text-classification", model=MODEL, top_k=3)
+    for text, preds in zip(texts, clf(texts)):
+        print(f"\n{text}")
+        for p in preds:
+            print(f"  {p['label']:64s} {p['score']:.3f}")
+
+
+if __name__ == "__main__":
+    main()

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}

train_state.json

@@ -0,0 +1 @@
+{"step": 700, "val_acc": 0.18725}