README.md

---
license: llama3.2
language:
- en
base_model: meta-llama/Llama-3.2-3B-Instruct
tags:
- construction-safety
- osha
- regulatory-compliance
- lora
- autoscientist
- adaption-labs
library_name: peft
---

# Llama 3.2 3B — Construction Code-Citation v2 (AutoScientist)

LoRA adapter on top of **Llama 3.2 3B-Instruct**, trained by **AutoScientist** from [Adaption Labs](https://adaptionlabs.ai) on the [construction-code-corpus-v1](https://huggingface.co/datasets/rigidhat/construction-code-corpus-v1) dataset. Predicts OIICS hazard codes (event, source, nature, body) and OSHA 29 CFR 1926 citations from construction-site incident narratives.

Built for the [Adaption Labs AutoScientist Challenge](https://adaptionlabs.ai/auto-scientist), **"All Other Domains"** category. Credit to **Adaptive Data by Adaption**.

## Story: Base Llama 3.2 3B vs AutoScientist-adapted

AutoScientist reported a **77% win rate** for the adapted model vs the base Llama 3.2 3B on the held-out task test set. Data adaption alone lifted quality **6.0 → 9.1** (grade C → A, +51.7% relative). The training recipe (rank/alpha/schedule/mixture) was chosen by AutoScientist end-to-end — not hand-tuned.

## Inputs / Outputs

Input: free-text construction-site incident narrative.
Output: strict JSON with `hazards[]` (4 OIICS codes + severity) and `citations[]` (verified OSHA 1926 standards).

```json
{
  "hazards": [
    {
      "code_event":  {"id": "21",   "title": "Slip or fall without fall to lower level"},
      "code_source": {"id": "5510", "title": "Ice, snow"},
      "code_nature": {"id": "220",  "title": "Fractures"},
      "code_body":   {"id": "280",  "title": "Trunk"},
      "severity": "high"
    }
  ],
  "citations": [
    {"standard": "1926.501", "section_heading": "Duty to have fall protection", "verified": true}
  ]
}
```

## Usage

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "meta-llama/Llama-3.2-3B-Instruct"
adapter = "rigidhat/llama-3.2-3b-construction-codecite-v2"

tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base)
model = PeftModel.from_pretrained(model, adapter)
```

For the full RAG-augmented prompt template + citation verifier, see `gradio_app/app.py` in the [source repo](https://github.com/snakezilla/construction-code-llm).

## Training recipe (from AutoScientist)

| Field | Value |
|---|---|
| Base model | meta-llama/Llama-3.2-3B-Instruct |
| Training method | SFT (LoRA) |
| LoRA rank | 32 |
| LoRA alpha | 64 |
| LoRA dropout | 0.0 |
| Target modules | all-linear (`gate_proj`, `k_proj`, `up_proj`, `down_proj`, `q_proj`, `o_proj`, `v_proj`) |
| Optimizer | AdamW (cosine LR, warmup 3%) |
| Peak LR | 2.0e-4 |
| Weight decay | 0.01 |
| Grad clip | 1.0 |
| Epochs | 3 |
| Steps | 66 |
| Train loss | 1.10 → 0.84 |
| Validation loss | 1.00 → 0.90 |

## Data

Trained on **[rigidhat/construction-code-corpus-v1](https://huggingface.co/datasets/rigidhat/construction-code-corpus-v1)** — 18,122 SFT records built from the OSHA Severe Injury Reports corpus (2015–2025), stratified 70/15/15 by NAICS subsector, with test split hash-pinned SHA-256 `c9490ed3...` and never opened during development.

Adaption applied: **Reasoning Traces**, **Hallucination Mitigation**, **Prompt Rephrase**, **Prompt Deduplication** via AutoScientist's data-adaption recipe.

## Demo

Live Gradio Space: **[rigidhat/construction-code-cite](https://huggingface.co/spaces/rigidhat/construction-code-cite)**

## Baseline for comparison

Same data, smaller manual QLoRA baseline: **[rigidhat/qwen-2.5-construction-codecite-v1](https://huggingface.co/rigidhat/qwen-2.5-construction-codecite-v1)** (Qwen 2.5 1.5B). Provided for methodological comparison — the v2 story is that AutoScientist end-to-end produced a stronger model than hand-tuned QLoRA.

## Source

Full pipeline (data pull, verifier, RAG index, eval harness): **[github.com/snakezilla/construction-code-llm](https://github.com/snakezilla/construction-code-llm)**

## License

Llama 3.2 Community License applies to the base model. LoRA adapter weights released under MIT.

## Citation

```
@misc{construction-codecite-v2-2026,
  title  = {Llama 3.2 3B - Construction Code-Citation v2 (AutoScientist)},
  author = {Oversite Innovations},
  year   = {2026},
  note   = {Trained by AutoScientist for the Adaption Labs AutoScientist Challenge, All Other Domains category}
}
```