PEFT
Safetensors
English
seismic
fault-interpretation
vision-language
geophysics
grounding
lora
thirdExec commited on
Commit
cf357e4
·
verified ·
1 Parent(s): 8ba4664

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +123 -0
README.md ADDED
@@ -0,0 +1,123 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-4.0
3
+ base_model: Qwen/Qwen2.5-1.5B-Instruct
4
+ datasets:
5
+ - thirdExec/synthetic-seismic-vlm
6
+ - GeoGPT-Research-Project/GeoGPT-CoT-QA
7
+ - gOLIVES/CRACKS
8
+ language:
9
+ - en
10
+ tags:
11
+ - seismic
12
+ - fault-interpretation
13
+ - vision-language
14
+ - geophysics
15
+ - grounding
16
+ - lora
17
+ - peft
18
+ ---
19
+
20
+ # SeisGround — trained weights
21
+
22
+ Vision–language weights for **seismic fault interpretation**: a frozen seismic encoder measures
23
+ fault geometry (count, location, dip, throw, per-fault mask); a language model then **copies those
24
+ measured numbers** into tagged narration through a non-differentiable digit seam, so the text can only
25
+ state values the vision reader actually measured.
26
+
27
+ **Pipeline:** `frozen SFM-Base-512 encoder → DETR reader (48 queries, Hungarian, ∅) → digit-copy seam → Qwen2.5-1.5B LM narrator (LoRA)`.
28
+
29
+ > ⚠️ **The encoder is NOT in this repo.** These weights run *on top of* the third-party **Seismic
30
+ > Foundation Model** (ViT-B/16 @512), which we use **frozen and unmodified**. Download it from the
31
+ > authors: [shenghanlin/SeismicFoundationModel](https://github.com/shenghanlin/SeismicFoundationModel#rocket-model-zoo-data-release)
32
+ > (Sheng et al., [arXiv:2309.02791](https://arxiv.org/abs/2309.02791)) and place it at
33
+ > `hybrid/checkpoints/SFM-Base-512.pth`.
34
+ >
35
+ > Code, scripts, and setup: **[github.com/Thirdbot/ModelV2](https://github.com/Thirdbot/ModelV2)** (see `SETUP.md`).
36
+
37
+ ## What each file is — and which result it backs
38
+
39
+ The repo mirrors the code's `hybrid/checkpoints/` tree, so `hf download … --local-dir hybrid/checkpoints`
40
+ reconstructs a runnable layout with no path surgery.
41
+
42
+ ### Main pipeline (synthetic training → deployable narration)
43
+ | File | What it is | Backs |
44
+ |---|---|---|
45
+ | `reader.pt` | Synthetic **base reader** — DETR set-prediction that measures fault count/location/dip/throw + per-fault mask. | Synthetic vision table (pooled IoU 0.230, det F1 0.433, class 0.93, dip 28.06° vs const 32.19°, throw 53.60 vs 56.23 ms) |
46
+ | `stage2_grounding.pt` | Grounding-stage LM adapter — teaches the narrator to read the digit seam. | (pipeline component) |
47
+ | `stage3_narrator.pt` | **Deployed narrator** (grounding+fuse LoRA) — free-generates tagged narration that copies the measured facts. | Faithfulness table (copy 0.77 GT-injected / 0.89 reader-piped; CHAIR$_I$ 0.185; dip-swap 16/16) |
48
+ | `stage3_answer.pt` | Answer-fold narrator variant (the `</think>→<answer>` fold). | (ablation / alternate narration) |
49
+ | `stage1_e12dcce6ed/` | **Geology LoRA adapter** — Qwen2.5-1.5B-Instruct, r16/α16, lr 2e-5, 4-bit, trained on GeoGPT-CoT-QA. Frozen thereafter; supplies the `<think>/<answer>` reasoning scaffold. | Geology stage-1 |
50
+
51
+ ### Real-field A/B + ratio-selection track (`ab_experiment/`)
52
+ Real adapters (r32, base frozen, zero-init residual) trained on top of a fresh synthetic base.
53
+ See `ab_experiment/PROVENANCE.md` for the full identity config.
54
+
55
+ | File | What it is | Backs |
56
+ |---|---|---|
57
+ | `ab_experiment/reader_synth.pt` | Synthetic reader **base** for this track (the frozen substrate the real adapters sit on). | — |
58
+ | `ab_experiment/B_joint.pt` | **DEPLOYED model.** 1:1:1 joint round-robin real adapter **with** real dip/throw supervision (`TRAIN_MEASURE=1`, data-gated to Smeaheia). | A/B table (B row) + all deployed inference |
59
+ | `ab_experiment/A_joint.pt` | Control — same 1:1:1, **no** real attribute supervision (`TRAIN_MEASURE=0`). Identical to B except the measurement heads never see real dip/throw. | A/B table (A row) |
60
+ | `ab_experiment/ratio1.pt`, `ratio2.pt` | The two **non-selected** mixing ratios (4:3:3 and 8:1:1) from ratio selection. 1:1:1 won on held-out val (mean det F1 0.363 vs 0.333 vs 0.166) and became A/B. | Ratio-selection table |
61
+ | `ab_experiment/alone_cracks.pt`, `alone_smeaheia.pt` | Single-survey **"alone" baselines** (no joint mixing). | Zero-shot / alone / joint table |
62
+
63
+ > If you release only a subset, keep the rows for the files you actually upload. `A_joint` is the
64
+ > paper's control; deployment needs only `B_joint` (+ `reader`, `stage3_narrator`, the geology adapter, and the SFM link).
65
+
66
+ ## How to use
67
+
68
+ ```bash
69
+ # 1) get the weights (mirrors hybrid/checkpoints/)
70
+ hf download thirdExec/seisground-weights --local-dir hybrid/checkpoints
71
+ # 2) add the frozen SFM encoder (third-party — see the note above) → hybrid/checkpoints/SFM-Base-512.pth
72
+ # 3) run inference (from the ModelV2 repo)
73
+ DATASET=synthetic python -m hybrid.eval.inference # in-distribution
74
+ DATASET=thebe READER=hybrid/checkpoints/ab_experiment/B_joint.pt python -m hybrid.eval.inference # a real survey
75
+ IMAGE=path/to/section.png READER=hybrid/checkpoints/ab_experiment/B_joint.pt python -m hybrid.infer # your own image
76
+ ```
77
+ The narrator defaults to `stage3_narrator.pt`; override with `CKPT=` / `NARRATOR=`.
78
+
79
+ ## Training data
80
+
81
+ | Dataset | Role | Source |
82
+ |---|---|---|
83
+ | Synthetic seismic VQA (1,261 scenes / 1,320 regions) | full supervision — masks, attributes, narration | [`thirdExec/synthetic-seismic-vlm`](https://huggingface.co/datasets/thirdExec/synthetic-seismic-vlm) |
84
+ | GeoGPT-CoT-QA | geology reasoning scaffold (stage 1) | [`GeoGPT-Research-Project/GeoGPT-CoT-QA`](https://huggingface.co/datasets/GeoGPT-Research-Project/GeoGPT-CoT-QA) |
85
+ | Thebe (37,796 patches) | real fault masks | Kaggle `mycarta/thebe-fault-patches-256` / Harvard Dataverse DOI [10.7910/DVN/YBYGBK](https://doi.org/10.7910/DVN/YBYGBK) |
86
+ | CRACKS (397) | real fault masks | [`gOLIVES/CRACKS`](https://huggingface.co/datasets/gOLIVES/CRACKS) |
87
+ | Smeaheia (430, GN1101 3-D cube) | real masks + independent dip/throw GT | [co2datashare.org](https://co2datashare.org/dataset/smeaheia-dataset) |
88
+
89
+ ## Identity config (defines the numbers)
90
+
91
+ Encoder SFM-Base-512 frozen (d768/depth12/patch16/img512/tile512/grid32) · loss Focal-Tversky
92
+ α0.4/β0.6/γ1.0, POS_WEIGHT_MAX 15, clDice 1.0 · N_QUERIES 48 · DET_TAU 0.1 · DET_THRESH 0.9 ·
93
+ **DILATE_R 0 (pure masks, 16px floor)** · geology LoRA r16/α16 lr2e-5 4-bit · reader lr1e-4
94
+ (encoder frozen) · grounding lr1e-4 · fold lr2e-5 · real adapter r32 lr1e-4 · ACTIVE_CLASSES=fault ·
95
+ TOTAL_STEPS 100000 (1 round-robin epoch) · **single seed 42**.
96
+
97
+ Trained on **RTX 3090 Ti 24 GB / 62 GB RAM**; Python 3.13, CUDA 12.8, torch 2.10, transformers 5.5,
98
+ peft 0.19, trl 0.21. The method is VRAM-agnostic — only compute-layout knobs (batch, grad-ckpt, cache
99
+ cap) change to fit smaller GPUs; the identity config, and therefore the result, does not move.
100
+
101
+ ## Provenance & honest limits
102
+
103
+ - **Single run, seed pinned** (`SEED=42` in the training entry points) — not multi-seed averaged.
104
+ Small-margin metrics (CRACKS det F1, dip parity, low-count Smeaheia) carry run variance; CUDA kernels
105
+ are not bit-deterministic.
106
+ - **Masks are pure (DILATE_R 0).** Segmentation numbers are at zero dilation; they are **not**
107
+ comparable to any dilated (DILATE_R > 0) evaluation.
108
+ - **Two Thebe builds are not comparable.** These A/B weights use the 37,796-patch Kaggle build (pure
109
+ masks). Do not compare their mask numbers to any older dilated-mask Thebe result.
110
+ - The narrator **copies** measured facts (digit seam); it does not regress numbers and cannot state a
111
+ value the reader did not measure. Out-of-domain prompts are answered against the on-screen seismic
112
+ scene, not as a general chatbot.
113
+
114
+ ## License
115
+
116
+ Weights released under **CC-BY-4.0** *(confirm/adjust before publishing)*. Derived from
117
+ Qwen2.5-1.5B-Instruct (Apache-2.0) via LoRA; respect the upstream licenses of the SFM encoder and each
118
+ training dataset. The SFM encoder itself is **not redistributed here** — obtain it from its original release.
119
+
120
+ ## Citation
121
+
122
+ If you use these weights, cite the paper (see the GitHub repo for the current reference) and the
123
+ underlying assets: the SFM encoder (Sheng et al., 2023), GeoGPT-CoT-QA, Thebe, CRACKS, and Smeaheia.