Update README with full training package documentation

README.md

@@ -20,124 +20,82 @@ pipeline_tag: text-generation
 library_name: peft
 ---
 
-# GIS-Coder: A Code Model for Geographic Information Systems
-
-**GIS-Coder** is a LoRA fine-tuned code model specialized in GIS (Geographic Information Systems) and geospatial analysis. Built on top of [Qwen2.5-Coder-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B-Instruct), it has been fine-tuned on expert-curated GIS code instruction data covering 13+ geospatial Python libraries.
-
-## 🗺️ What Can GIS-Coder Do?
-
-- **OSMnx**: Road network analysis, routing, isochrones, POI extraction from OpenStreetMap
-- **GeoPandas**: Spatial joins, buffering, dissolving, geocoding, file I/O
-- **Rasterio**: Raster reprojection, NDVI calculation, zonal statistics, hillshade
-- **Shapely**: Geometry creation, boolean operations, validation, simplification
-- **GDAL/OGR**: Raster merging, clipping, format conversion
-- **PyProj**: CRS conversions, UTM zone detection, coordinate transformations
-- **Folium**: Interactive web maps, heatmaps, choropleths, marker clusters
-- **MovingPandas**: GPS trajectory analysis, stop detection, generalization
-- **H3**: Hexagonal spatial indexing, density aggregation
-- **Fiona**: Low-level vector I/O, filtering, format conversion
-- **xarray/rioxarray**: Climate data analysis, NetCDF processing, raster export
-- **PyQGIS**: QGIS scripting, processing algorithms, map layouts
-- **PySAL**: Spatial autocorrelation (Moran's I), LISA clusters
-
-## 📊 Training Details
-
-| Metric | Value |
-|--------|-------|
-| **Base Model** | Qwen/Qwen2.5-Coder-0.5B-Instruct (494M params) |
-| **Method** | LoRA SFT (r=8, α=16, target: q/k/v/o/gate/up/down_proj) |
-| **Dataset** | 30 expert-curated GIS code instruction pairs with CoT |
-| **Training Loss** | 1.52 → 0.88 (−42% over 3 epochs) |
-| **Token Accuracy** | 69.3% → 79.3% (+10pp) |
-| **Evaluation Score** | 85% (code + GIS lib + CoT + function quality) |
-
-### Training Recipe
-
-Based on research from:
-- **CFD fine-tuning paper** (arxiv:2504.09602): LoRA SFT recipe that outperformed 72B models with a 7B model
-- **MapCoder-Lite** (arxiv:2509.17489): Qwen2.5-Coder as best backbone for domain code tasks
-- **GIS benchmark** (arxiv:2410.04617): Identified critical gaps in OSMNX, MovingPandas, Rasterio coverage
-
-### Hyperparameters
-
-```yaml
-learning_rate: 2e-4
-lr_scheduler: cosine
-warmup_ratio: 0.1
-epochs: 3
-batch_size: 1 (gradient_accumulation_steps: 4)
-max_length: 1024
-lora_r: 8
-lora_alpha: 16
-lora_dropout: 0.05
-```
+# GIS-Coder — A Code Model for Geographic Information Systems
+
+A LoRA-adapted code model specialized for GIS and geospatial Python programming. Includes a **ready-to-run training package** for scaling up to 7B on your own GPU cluster.
+
+## 📦 This Repo Contains
+
+| File | Description |
+|------|-------------|
+| `adapter_model.safetensors` | Trained LoRA adapter (0.5B base, proof of concept) |
+| `train_7b.py` | **Production 7B QLoRA training script** with CLI args |
+| `evaluate.py` | Evaluation suite (12 GIS benchmarks with scoring) |
+| `requirements.txt` | All dependencies |
+| `TRAINING_README.md` | **Detailed training guide** — hardware, hyperparameters, ablations |
 
-## 🚀 Usage
+## 🚀 Train the 7B Model on Your GPUs
 
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
-from peft import PeftModel
+```bash
+# 1. Clone this repo
+git clone https://huggingface.co/RhodWeo/GIS-Coder-7B
+cd GIS-Coder-7B
 
-# Load base model + adapter
-base_model = AutoModelForCausalLM.from_pretrained(
-    "Qwen/Qwen2.5-Coder-0.5B-Instruct",
-    trust_remote_code=True,
-)
-model = PeftModel.from_pretrained(base_model, "RhodWeo/GIS-Coder-7B")
-tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-0.5B-Instruct")
+# 2. Install deps
+pip install -r requirements.txt
 
-# Create prompt
-messages = [
-    {"role": "system", "content": "You are GIS-Coder, an expert Python programmer specializing in GIS and geospatial analysis."},
-    {"role": "user", "content": "Write a function to calculate NDVI from a satellite image using rasterio."}
-]
+# 3. Login
+huggingface-cli login
 
-text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
-inputs = tokenizer(text, return_tensors="pt")
+# 4. Train! (A100 80GB recommended)
+python train_7b.py
 
-outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, top_p=0.9)
-print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+# For A10G/RTX 4090 (24GB):
+python train_7b.py --batch_size 1 --grad_accum 16 --max_length 2048
+
+# For H100:
+python train_7b.py --batch_size 4 --grad_accum 4 --max_length 8192
+
+# 5. Evaluate
+python evaluate.py --adapter_id ./gis-coder-7b-output/final --compare_base
 ```
 
-## 📈 Training Curves
+See **[TRAINING_README.md](TRAINING_README.md)** for the full guide with hardware-specific settings, ablation ideas, and expected results.
 
-| Epoch | Loss | Token Accuracy |
-|-------|------|----------------|
-| 0.13 | 1.520 | 69.3% |
-| 1.00 | 1.238 | 72.2% |
-| 2.00 | 1.007 | 75.5% |
-| 3.00 | 0.880 | **79.3%** |
+## 🗺️ GIS Libraries Covered (13)
 
-## 🔬 Evaluation Results
+| Priority | Libraries | Coverage |
+|----------|-----------|----------|
+| **Tier 1** (0% baseline) | OSMnx, MovingPandas, Rasterio, GDAL, PyProj | Heavy — these are where models fail |
+| **Tier 2** | GeoPandas, Shapely, H3 | Core GIS operations |
+| **Tier 3** | Folium, xarray, PyQGIS, Fiona, PySAL | Real-world workflows |
 
-Tested on 5 GIS code generation prompts covering OSMnx, Rasterio, GeoPandas, CRS handling, and multi-library workflows:
+## 📊 Proof-of-Concept Results (0.5B)
 
-| Metric | Score |
-|--------|-------|
-| Code blocks generated | 100% |
-| Correct GIS library usage | 100% |
-| Chain-of-thought reasoning | 60% |
-| Function definitions | 80% |
-| **Overall Quality** | **85%** |
+Trained on CPU with the smaller base model to validate the approach:
 
-## 📚 Dataset
+| Metric | Start → End |
+|--------|------------|
+| **Loss** | 1.52 → 0.88 (−42%) |
+| **Token Accuracy** | 69.3% → **79.3%** (+10pp) |
+| **Eval Quality** | **85%** (code + library + CoT + function) |
 
-Training dataset: [RhodWeo/gis-code-instructions](https://huggingface.co/datasets/RhodWeo/gis-code-instructions)
+## 🔬 Training Recipe
 
-30 expert-curated instruction pairs covering:
-- Tier 1 (models score 0%): OSMnx, MovingPandas, Rasterio, GDAL, PyProj
-- Tier 2 (partial coverage): GeoPandas, Shapely, H3
-- Tier 3 (workflow): Folium, xarray, PyQGIS, Fiona, PySAL
+Based on published research:
 
-## ⚡ Scaling Up
+| Principle | Source | Applied |
+|-----------|--------|---------|
+| QLoRA SFT beats 72B models | [CFD paper](https://arxiv.org/abs/2504.09602) | r=32, all-linear, lr=2e-4 |
+| Qwen2.5-Coder best backbone | [MapCoder-Lite](https://arxiv.org/abs/2509.17489) | Base model selection |
+| Models score 0% on GIS | [GIS Benchmark](https://arxiv.org/abs/2410.04617) | Heavy OSMnx/MovingPandas coverage |
+| CoT boosts +20.9% pass@1 | CFD paper ablation | All examples include CoT |
+| Target all linear layers | [LoRA Without Regret](https://arxiv.org/abs/2410.13732) | `target_modules="all-linear"` |
 
-This model was trained on CPU with a 0.5B parameter base. For production use, we recommend:
+## 📚 Dataset
 
-1. **Scale the base model**: Use `Qwen/Qwen2.5-Coder-7B-Instruct` with QLoRA on A100 GPU
-2. **Scale the dataset**: Generate 20K+ examples using OSS-Instruct (Magicoder) pattern with GIS code as seeds
-3. **Add execution-based filtering**: Test all generated code and keep only passing examples
-4. **Include CoT annotations**: +20.9% pass@1 improvement per CFD paper ablation
+**[RhodWeo/gis-code-instructions](https://huggingface.co/datasets/RhodWeo/gis-code-instructions)** — 70 expert-curated examples with Chain-of-Thought annotations.
 
 ## License
 
-Apache 2.0 (same as base model)
+Apache 2.0