rigidhat commited on
Commit
2e674e0
·
verified ·
1 Parent(s): b80e93c

Upload folder using huggingface_hub

Browse files
Files changed (3) hide show
  1. README.md +94 -0
  2. adapter_config.json +41 -0
  3. adapters.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,94 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ language:
4
+ - en
5
+ base_model: mlx-community/Qwen2.5-1.5B-Instruct-4bit
6
+ tags:
7
+ - construction-safety
8
+ - osha
9
+ - regulatory-compliance
10
+ - lora
11
+ - mlx
12
+ library_name: mlx-lm
13
+ ---
14
+
15
+ # Qwen 2.5 1.5B — Construction Code-Citation v1
16
+
17
+ LoRA adapter on top of Qwen 2.5 1.5B-Instruct (4-bit MLX). Predicts OIICS
18
+ hazard codes (event, source, nature, body) and OSHA 29 CFR 1926 citations
19
+ from construction-site incident narratives.
20
+
21
+ Built for the [Adaption Labs AutoScientist Challenge](https://adaptionlabs.ai/auto-scientist)
22
+ ("All Other Domains" category).
23
+
24
+ ## Inputs / Outputs
25
+
26
+ Input: free-text construction-site narrative.
27
+ Output: strict JSON with `hazards[]` (4 OIICS codes + severity) and
28
+ `citations[]` (verified OSHA 1926 standards).
29
+
30
+ ## Usage
31
+
32
+ ```python
33
+ from mlx_lm import load, generate
34
+
35
+ model, tokenizer = load(
36
+ "mlx-community/Qwen2.5-1.5B-Instruct-4bit",
37
+ adapter_path="oversite/qwen-2.5-construction-codecite-v1",
38
+ )
39
+ prompt = "Worker fell from second-story scaffold..."
40
+ out = generate(model, tokenizer, prompt=prompt, max_tokens=384)
41
+ ```
42
+
43
+ See `gradio_app/app.py` in the source repo for the full prompt template
44
+ and RAG-augmented inference pipeline.
45
+
46
+ ## Training
47
+
48
+ - **Base model:** Qwen 2.5 1.5B-Instruct, 4-bit MLX quantization
49
+ - **Method:** LoRA, 16 layers, 5.3M trainable parameters
50
+ - **Data:** 17,127 stratified-by-event-division SFT examples from OSHA SIR
51
+ (2015-2025)
52
+ - **Optimizer:** Adam, lr 1e-4, batch 2, 400 iterations
53
+ - **Loss masking:** prompt masked, train on completion tokens only
54
+ - **Seed:** 20260606
55
+
56
+ ## Metrics
57
+
58
+ | Dimension | Accuracy |
59
+ |---|---|
60
+ | event_acc | 35.5% |
61
+ | event_div_acc | 48.0% |
62
+ | source_acc | 51.0% |
63
+ | source_div_acc | 33.0% |
64
+ | nature_acc | 66.5% |
65
+ | body_acc | 57.5% |
66
+ | body_div_acc | 87.5% |
67
+ | parsed_ok_rate | 100.0% |
68
+
69
+ _n=200, split=dev, git_sha=d926d81_
70
+
71
+ Test-set numbers are held back until submission per the locked split
72
+ (SHA-256 `c9490ed3...`).
73
+
74
+ ## Limitations
75
+
76
+ - Source-code distribution has a heavy long tail (1,478 unique codes).
77
+ Model uses an OTHER bucket for codes outside the top-75 shortlist.
78
+ - Citation grounding is BM25-only at v1 (vector index follow-up).
79
+ - SIR over-represents severe injuries; the model is biased toward
80
+ high-severity event types.
81
+
82
+ ## License
83
+
84
+ MIT. Base model Qwen 2.5 1.5B-Instruct is governed by its upstream license.
85
+
86
+ ## Citation
87
+
88
+ ```
89
+ @misc{construction-code-llm-2026,
90
+ title = {Qwen 2.5 1.5B - Construction Code-Citation v1},
91
+ author = {Oversite Innovations},
92
+ year = {2026}
93
+ }
94
+ ```
adapter_config.json ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "adapter_path": "train/runs/v1_qlora_001_cite",
3
+ "batch_size": 2,
4
+ "clear_cache_threshold": 0,
5
+ "config": null,
6
+ "data": "train/data",
7
+ "fine_tune_type": "lora",
8
+ "grad_accumulation_steps": 1,
9
+ "grad_checkpoint": false,
10
+ "iters": 400,
11
+ "learning_rate": 0.0001,
12
+ "lora_parameters": {
13
+ "rank": 8,
14
+ "dropout": 0.0,
15
+ "scale": 20.0
16
+ },
17
+ "lr_schedule": null,
18
+ "mask_prompt": true,
19
+ "max_seq_length": 2048,
20
+ "model": "mlx-community/Qwen2.5-1.5B-Instruct-4bit",
21
+ "num_layers": 16,
22
+ "optimizer": "adam",
23
+ "optimizer_config": {
24
+ "adam": {},
25
+ "adamw": {},
26
+ "muon": {},
27
+ "sgd": {},
28
+ "adafactor": {}
29
+ },
30
+ "project_name": null,
31
+ "report_to": null,
32
+ "resume_adapter_file": null,
33
+ "save_every": 100,
34
+ "seed": 20260606,
35
+ "steps_per_eval": 100,
36
+ "steps_per_report": 25,
37
+ "test": false,
38
+ "test_batches": 500,
39
+ "train": true,
40
+ "val_batches": 25
41
+ }
adapters.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e7b619fe5d48b5a3af1a4d5742c0460aef74d5bf20cfb01c6815620549da778d
3
+ size 21126646