PROJECT.md

Local First Education Data Framework — Project Knowledge Base

Last updated: 2026-06-14 (Session 7 — UI polish, CSV download, about modal, percentage fix, rebrand) Target: HF Build Small Hackathon — "Backyard AI" chapter Space: https://huggingface.co/spaces/build-small-hackathon/Kasualdad_LFED Remote: git@hf.co:spaces/build-small-hackathon/Kasualdad_LFED (remote name: space)

Note: the Space slug remains Kasualdad_LFED for HF URL stability; the product name is now Local First Education Data Framework (LFED).

---

Current State (read this first)

main is the live HF Space demo. It runs transformers + PEFT on ZeroGPU: a bnb-4-bit base (unsloth/qwen2.5-coder-14b-instruct-bnb-4bit) plus the fine-tuned LoRA adapter (build-small-hackathon/lfed-qwen2.5-coder-14b-sql-lora).

The local-first product path lives on product (branch) / local-llamacpp-v1 (tag) and uses llama.cpp + GGUF. The main branch and the product branch share the same fine-tuned weights but use different inference engines because ZeroGPU only exposes PyTorch CUDA.

Recent changes (2026-06-14)

• UI polish complete — removed dark wrapper backgrounds, fixed dataframe light theme, darkened labels, aligned the three domain dropdown columns.
• "Show me how this was computed" SQL disclosure — added below each result table.
• CSV download — result tables can be downloaded as kasualdad_result_YYYYMMDD_HHMMSS.csv.
• About / FAQ modal — "First time here?" opens an overlay with intro, how-it-works, what it is/isn't, FAQ, and privacy.
• "Bring it back" button hidden — logic preserved but button is always invisible; previous-answer ribbon text remains.
• Percentage formatting fixed — format_result_df() no longer double-multiplies values that are already returned as percentages.
• Evaluation query bank — evaluation_queries.md added to the repo for local testing (not pushed to the Space).

---

What It Does

A Gradio app that lets school district staff ask plain-English questions about student data. A fine-tuned LLM (Qwen2.5-Coder-14B, QLoRA on 27,859 NL→SQL pairs) generates DuckDB SQL, which is validated and executed on in-memory seed data. Results are returned as a plain-English summary, a table, an optional CSV download, and the generated SQL.

On the product branch nothing leaves the machine; the Space demo runs the same fine-tune on ZeroGPU.

---

Architecture & Data Flow

User types question or picks a starter dropdown
  ↓
app.py  →  on_submit()  →  handle_query()  [@spaces.GPU on ZeroGPU]
  ↓
model_inference.py  →  build_prompt() + TransformersLLM (bnb-4bit base + LoRA)
  ↓  streamed SQL tokens (llama.cpp-compatible response schema)
data_engine.py  →  extract_sql() → validate_sql() → execute_safe()
  ↓  pandas DataFrame
app.py  →  renders summary, table, CSV link, and SQL disclosure

Key design rule: app.py is a thin controller. All logic lives in the engine modules.

UI Layout (current)

1. Header — title + tagline
2. Input box + "Get answer" button
3. Three domain starter dropdowns (attendance, grades, discipline/enrollment)
4. First-time nudge with modal link
5. Result region — previous-answer ribbon, summary, dataframe, CSV download, SQL disclosure
6. Footer — what this is / what it isn't + "Read the full explainer"
7. Hidden about/FAQ modal overlay

---

File Map

File	Purpose	Lines (approx.)
app.py	Gradio UI, streaming handle_query generator, @spaces.GPU, Parquet bootstrap, custom CSS, about modal, CSV download	~1,250
model_inference.py	TransformersLLM wrapper (bnb-4bit base + LoRA), SQL generation + streaming	~270
data_engine.py	DuckDB lifecycle, Parquet loading, SQL extraction/validation/execution	~280
prompts.py	System prompt, 5-table schema docs, 4 few-shot examples, prompt assembler	~190
ui_strings.py	All user-facing copy: titles, nudges, domain starter questions, about/FAQ content	~230
data/generate_seed.py	Deterministic generator: ~2,900 students across 5 tables	~430
data/export_parquet.py	One-shot script: seed → 5 Parquet files	~70
data/*.parquet	5 committed seed files via LFS (~260 KB total, byte-deterministic)	—
requirements.txt	Pinned Python deps	9 lines
README.md	Public Space README	~280
PROJECT.md	This file — project knowledge base	~500
evaluation_queries.md	15 real-world evaluation queries for manual testing	~60
docs/HANDOFF.md	Developer session handoff doc	~190
docs/PLAN.md	Original build plan	~170
docs/SPEC_query-history-dashboards.md	Draft spec for next features	—
tests/	pytest suite (81 tests)	4 files
modal_train/	Modal fine-tuning pipeline	5+ files

---

Model & Inference

Active Model (main — Space)

• Base: unsloth/qwen2.5-coder-14b-instruct-bnb-4bit (pre-quantized NF4, ~10 GB download)
• Adapter: build-small-hackathon/lfed-qwen2.5-coder-14b-sql-lora (551 MB, QLoRA r=32, α=32)
• Loaded via: transformers AutoModelForCausalLM + PeftModel.from_pretrained(..., torch_device="cpu") (required on ZeroGPU)
• Env overrides: LFED_BASE_MODEL, LFED_ADAPTER_REPO

Active Model (product — local) [HISTORICAL — llama.cpp]

• Repo: build-small-hackathon/lfed-qwen2.5-coder-14b-sql-gguf (Q4_K_M, ~9 GB; 7B fallback exists)
• llama.cpp via llama-cpp-python, Metal on macOS

Inference Config (main)

Parameter	Value	Note
max_tokens	192	Outputs run ~140-170 chars
temperature	0.0	Deterministic (greedy)
stop	\n\n, Question:, User:, <|im_end|>, <|im_start|>	Applied post-hoc + streaming truncation
Few-shots	4	Trimmed from 7 to cut bnb-4bit prefill cost
spaces.GPU	duration=30	Shorter duration = ZeroGPU queue priority
Generation time	~5 s/query	Rest of latency = ZeroGPU queue + weight restore

Thread Safety

model_inference.py caches the model in a module-level _llm global with a threading.Lock (double-check pattern). generate_sql() auto-loads if llm=None.

---

Zero GPU / HF Space Configuration

Space Identity

• URL: https://huggingface.co/spaces/build-small-hackathon/Kasualdad_LFED
• SDK: Gradio 6.17.3
• Python: 3.12
• Hardware: Zero GPU (NVIDIA RTX Pro 6000 Blackwell, half — 48 GB VRAM)
• Git remote: space → https://huggingface.co/spaces/build-small-hackathon/Kasualdad_LFED

Zero GPU Architecture

Zero GPU uses CUDA emulation at module level and real GPU inside @spaces.GPU functions. This means:

• Model loads with n_gpu_layers=-1 at startup (module level, emulated CUDA)
• Inference runs inside @spaces.GPU decorated handle_query() (real GPU)
• Zero GPU transparently switches between emulated and real CUDA contexts

@spaces.GPU Decorator

app.py — handle_query() is the only decorated function. Required for Zero GPU to recognize the Space as GPU-enabled.

Startup Sequence (app.py)

1. Print banner
2. Ensure Parquet seed files exist — all 5 files (enrollment, attendance, students, discipline, grades) must be present in /data/ or data/; they are committed via LFS (byte-deterministic). If any are missing, regenerate via export_parquet.py.
3. Load model (load_model()) — downloads base + adapter from HF Hub if not cached
4. Launch Gradio UI

---

Data Engine & Parquet Optimization

Per-Request Lifecycle

data_engine.py:create_session() → get_connection() + seed_database()

Each query creates a fresh in-memory DuckDB connection. This ensures:

• Thread safety (no shared state between requests)
• Query isolation (can't affect other requests)
• Clean state (no stale data)

Seeding Priority

data_engine.py:seed_database()

1. Parquet files (fastest — ~260 KB, single-digit ms). Requires all 5 tables in /data/ (Space persistent storage) or data/ (local dev, committed via LFS). Loaded via DuckDB read_parquet() → CREATE TABLE ... AS SELECT *.
2. Python generator data/generate_seed.py (slow fallback — ~2,900 students)

Parquet Bootstrap

app.py — On startup, if no Parquet files are found in any of the search dirs, export_parquet.py is called to generate them. On the Space, they go to /data/ which persists across restarts.

SQL Safety Pipeline

data_engine.py:execute_safe()

1. extract_sql() — Parse JSON envelope → sql block → raw fallback
2. validate_sql() — Forbidden token check (DROP, DELETE, INSERT, UPDATE, etc.) + schema-aware EXPLAIN validation
3. Wrap: SELECT * FROM (<user_query>) AS _safe LIMIT 1000
4. Execute directly on DuckDB
5. Return (cleaned_sql, DataFrame)

Forbidden Tokens

data_engine.py — DROP, DELETE, INSERT, UPDATE, ALTER, TRUNCATE, CREATE, ATTACH, DETACH, PRAGMA

---

Database Schema

enrollment

Column	Type	Description
school_year	VARCHAR	School year, format 'YYYY-YYYY'
school_name	VARCHAR	One of 5 schools
grade_level	INTEGER	Grade level (K=0 through 12)
student_count	INTEGER	Students enrolled in that grade/year/school

attendance

Column	Type	Description
student_id	INTEGER	Unique student identifier
school_name	VARCHAR	School the student attends
school_year	VARCHAR	School year, format 'YYYY-YYYY'
absence_count	INTEGER	Total absences for that year
is_chronically_absent	BOOLEAN	TRUE if missed ≥10% of school days

students

Column	Type	Description
student_id	INTEGER	Unique student identifier
school_name	VARCHAR	School the student attends
grade_level	INTEGER	Grade level (K=0 through 12)
gender, race_ethnicity	VARCHAR	Demographic fields
english_learner, special_education, economically_disadvantaged	BOOLEAN	Program flags

discipline

Column	Type	Description
incident_id, student_id	INTEGER	IDs
school_name, school_year	VARCHAR	School / year
grade_level	INTEGER	Grade at time of incident
incident_type, severity, action_taken	VARCHAR	Incident details
incident_date	DATE	Date of incident
days_suspended	INTEGER	Days suspended, if any

grades

Column	Type	Description
student_id	INTEGER	Student ID
school_name, school_year	VARCHAR	School / year
grade_level	INTEGER	Grade level
course_name, term, letter_grade	VARCHAR	Course details
grade_numeric, gpa	DOUBLE	Numeric grade and GPA

Schools

School	Grades	Base Enrollment
Lincoln Elementary	K–5	520
Washington Middle	6–8	480
Jefferson High	9–12	900
Roosevelt Academy	K–8	380
Kennedy Prep	6–12	620

Seed Data Stats

• Students: ~2,900
• Chronic absenteeism rate: ~15%
• Enrollment rows: ~116
• Attendance rows: ~11,600
• Discipline rows: ~1,400
• Grades rows: ~160,000
• School years: 2021-2022, 2022-2023, 2023-2024, 2024-2025
• Reproducible: random.seed(42) in generate_seed.py

---

Prompt Engineering

prompts.py assembles the full LLM prompt:

Structure

SYSTEM_PROMPT (rules, constraints, output format)
  ↓
Schema documentation (table + column list)
  ↓
Few-shot examples (4 question→SQL pairs)
  ↓
User question
  ↓
"SQL:"

System Prompt Rules

1. Only SELECT statements
2. Exact table/column names from schema
3. Proper DuckDB syntax (VARCHAR → single quotes, BOOLEAN → TRUE/FALSE)
4. Use column aliases for aggregations
5. Join on logical columns
6. Make reasonable assumptions if ambiguous
7. Output ONLY the sql block, no explanation

Few-Shot Examples

Question	SQL pattern
"How many chronically absent in 2023-2024?"	COUNT(*) with WHERE ... AND is_chronically_absent = TRUE
"Show enrollment per school sorted"	SUM(student_count) GROUP BY school_name ORDER BY ... DESC
"What percentage at Lincoln Elementary?"	COUNT(CASE WHEN ...) * 100.0 / COUNT(*)
"Enrollment trend since 2021"	GROUP BY school_year ORDER BY school_year

---

Fine-Tuning Pipeline (Modal)

Location: modal_train/ — v2

Script	Purpose
generate_synthetic_v2.py + modal_generate.py + augment_gretel.py + rephrase_pairs.py	Builds the 27,859-pair dataset (train_final_v2.jsonl)
train_v2.py	Unsloth QLoRA on Qwen2.5-Coder-14B (r=32, α=32, 4-bit, 2 epochs, lr=1e-4, A10G)
export_gguf_v2.py	Merges LoRA → GGUF Q4_K_M → pushes to HF Hub
modal_train_v2.py	Modal orchestration (modal.App("kasualdad-lfed-train-v2"))

Published training dataset

The v2 training dataset is now published on the Hub:

Artifact	Location
NL→SQL training data (25,886 pairs)	build-small-hackathon/lfed-training-data
v1 scripts (train.py, export_gguf.py, modal_app.py)	Historical 7B run, 1,289 pairs

How to Run

modal secret create huggingface-secret HF_TOKEN=<token>
modal run modal_train/modal_train_v2.py

Artifacts (Status: trained 2026-06-10)

Artifact	Location
LoRA adapter	build-small-hackathon/lfed-qwen2.5-coder-14b-sql-lora (HF Hub) + Modal volume lfed-training-data:/lora-adapter-v2
GGUF Q4_K_M	build-small-hackathon/lfed-qwen2.5-coder-14b-sql-gguf (HF Hub)
Merged fp16	Not persisted; re-merge from base + adapter if needed

---

Dependencies

Python (requirements.txt, main branch)

Package	Why
gradio >= 6.15	Web UI
duckdb 1.5.3	In-memory SQL engine
torch / transformers / peft / bitsandbytes / accelerate	Model inference (bnb-4bit base + LoRA)
huggingface_hub + hf_transfer	Fast model download
spaces	@spaces.GPU decorator for ZeroGPU

The product branch instead pins llama-cpp-python (no torch stack needed).

---

UI Design System

Current design is ResearchMono on top of gr.themes.Soft:

• Typography: IBM Plex Sans (UI) + IBM Plex Mono (SQL/code)
• Palette: light page background (#F2F4F8), white surfaces (#FFFFFF), dark text (#21272A), IBM blue accent (#4589FF)
• Layout: single column; result region appears after submitting a question
• Components: domain starter dropdowns, first-time nudge + modal, result summary, dataframe, CSV download, SQL disclosure, footer explainer

The previous aggressive CSS-override approach was replaced by gr.themes.Soft with targeted overrides for Gradio's scoped Svelte classes.

---

Hackathon Badges

Badge	Status	Implementation
Off the Grid	✅	product branch: llama.cpp + local GGUF, no API calls. main/Space: same fine-tune on ZeroGPU (transformers).
Well-Tuned	✅	Fine-tuned Qwen2.5-Coder-14B, QLoRA r=32 on 27,859 NL→SQL pairs (Modal A10G).
Llama Champion	✅	llama.cpp is the local-first inference backend (GGUF Q4_K_M, Metal, streaming).
Off-Brand	✅	Custom ResearchMono theme — IBM Plex Sans/Mono, #4589FF accent, gr.themes.Soft.

---

Gotchas & Pitfalls

1. LD_LIBRARY_PATH doesn't work at Python runtime [HISTORICAL — llama.cpp]

The dynamic linker caches it at process start. Use ctypes.CDLL(path, mode=RTLD_GLOBAL) to preload libraries instead. Only relevant for the product branch.

2. ZeroGPU = PyTorch-only CUDA [2026-06-12]

ZeroGPU's spaces.GPU decorator works through a PyTorch patching layer — non-PyTorch CUDA libraries (llama.cpp) cannot benefit from it. The fix was to drop llama.cpp on the Space and use transformers + PEFT + bnb-4-bit instead.

3. PEFT adapter loading straight to cuda:0 fails on ZeroGPU [2026-06-12]

PeftModel.from_pretrained() defaults to loading adapter safetensors directly onto cuda:0, which ZeroGPU forbids at startup. Fix: torch_device="cpu".

4. Parquet files committed via LFS [2026-06-12]

All 5 Parquet seed files are committed via git LFS. They are byte-deterministic. HF Spaces requires LFS for binary files >10 KB. Re-export with python data/export_parquet.py if the seed generator changes.

5. Missing Parquet fallback = nondeterministic data per query [2026-06-12]

seed_database() requires all 5 Parquet files. If any are missing, it falls through to the Python generator, which advances the shared random RNG on each call → different data per query. Fixed by committing all 5 files + re-seeding the RNG inside generate_seed_data().

6. Gradio scoped Svelte CSS beats generic selectors [2026-06-14]

Generic selectors like .gradio-container .form or .gr-dataframe are overridden by Gradio's generated scoped classes (e.g., .form.svelte-d5xbca). Fixes required inspecting the live DOM and using higher-specificity selectors, !important, or overriding CSS variables.

7. Percentage formatting double-scaling [2026-06-14]

format_result_df() originally multiplied every _rate column by 100, assuming the model always returned 0–1 proportions. The model sometimes returns percentages already (e.g., 10.0), causing values like 1000.00%. Fixed by scaling only values in the 0 <= v <= 1 range.

8. Transient HF Spaces mount errors [2026-06-14]

The Space occasionally fails during init with Initialization step 'hf-mount' failed. This is an HF infrastructure issue attaching the persistent storage bucket, not an app bug. Restarting the Space from the Hugging Face UI usually resolves it; the app also has a local Parquet fallback.

---

Quick Start

# Clone
cd Local-First-Education-Data-Framework

# Virtual env
python3.12 -m venv .venv && source .venv/bin/activate

# Install
pip install -r requirements.txt

# Generate Parquet seed files (first time only)
python data/export_parquet.py

# Run
python app.py
# → http://localhost:7860

# Tests
pytest tests/ -v

# Deploy
git push space main

---

Recent Git History

549e4e0 fix: only scale proportions 0-1 to percentages; avoid double-multiplying
d6754e1 feat: hide bring-back button, add CSV download, about/FAQ modal
432039e fix: target real gr.Code label, explainer block border, align dropdown columns
95ddb2d fix: dark sql disclosure text, remove explainer border
78ebc5c fix: override Gradio scoped .form background, label color, dataframe table vars

---

Future Work

• Fine-tuning completed — 14B model trained (27,859 pairs), GGUF + LoRA adapter published
• Swapped to fine-tuned model on main — transformers + LoRA on ZeroGPU
• Streaming re-enabled — SQL streams to the UI
• Deterministic seed data — all 5 Parquet files committed via LFS
• UI polish — theme, labels, borders, dataframe, dropdown alignment
• CSV download
• About / FAQ modal
• Evaluation query bank
• Query history + comparisons — see docs/SPEC_query-history-dashboards.md
• Dashboards — Standard Board + Ephemeral Scratch Board
• HF Space smoke test — verify a subset of evaluation queries after each deploy
• Product branch development — build local-first features on product branch
• Model card update on build-small-hackathon/lfed-qwen2.5-coder-14b-sql-lora

---

Historical Sections

Sections below are kept for reference but describe earlier versions of the project.

UI Design System (pre-2026-06-10)

The original palette used slate/indigo tones with aggressive CSS overrides against Gradio's dark-theme defaults. It was replaced by the current gr.themes.Soft + IBM Plex ResearchMono approach.

CUDA Dependency Story (llama.cpp backend)

Full debugging history of preloading CUDA shared libraries for llama-cpp-python on the HF Space. Only relevant to the product branch. The main branch no longer includes llama.cpp or the CUDA preload code.

llama.cpp Import Guard

If from llama_cpp import Llama failed on the old backend, a RuntimeError pointed users to the CPU wheel. This code was removed on main in 2026-06-12.