rishabhsai commited on
Commit
57a67db
·
verified ·
1 Parent(s): b303092

Codex: upload Offline Form Pilot Space

Browse files
.gitignore ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ .venv/
2
+ __pycache__/
3
+ *.pyc
4
+ .pytest_cache/
5
+ .ruff_cache/
6
+
7
+ traces/*.json
8
+ traces/*.jsonl
9
+ exports/*.json
10
+ exports/*.csv
11
+ *.log
12
+
13
+ .env
14
+ .DS_Store
CODEX_BUILD_LOG.md ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Codex Build Log
2
+
3
+ This project is being built with OpenAI Codex as the coding agent for the Build Small Hackathon OpenAI/Codex track.
4
+
5
+ ## Build Principles
6
+
7
+ - Keep commits small and Codex-attributed.
8
+ - Keep the app review-first: never auto-submit forms.
9
+ - Make model behavior inspectable through structured outputs and traces.
10
+ - Prefer a small-model-friendly workflow over a generic chatbot.
11
+
12
+ ## Timeline
13
+
14
+ ### 2026-06-06
15
+
16
+ - User selected the form-filling concept.
17
+ - Codex initialized a fresh standalone project under `projects/offline-form-pilot`.
18
+ - Codex added the initial README, build log, requirements, and git hygiene files.
19
+ - Codex implemented the first Gradio app, local structured matcher, optional Hugging Face small-model assist, trace export, and engine tests.
20
+ - Codex ran `pytest` in the project venv and a direct app smoke check.
21
+ - Codex created and pushed the public GitHub repo: https://github.com/rishabhsai/offline-form-pilot
22
+ - Codex confirmed the repo is public and added Hugging Face deployment notes plus a synthetic anonymized trace example.
23
+ - Codex attempted to create the official `build-small-hackathon/offline-form-pilot` Space, but HF returned a 403 for missing namespace creation rights.
24
+ - Codex created the fallback public personal Space: https://huggingface.co/spaces/rishabhsai/offline-form-pilot
25
+
26
+ ## Planned Codex Milestones
27
+
28
+ 1. Scaffold repo and README.
29
+ 2. Implement Gradio app and deterministic field matcher.
30
+ 3. Add optional small-model backends and trace export.
31
+ 4. Add tests and local verification.
32
+ 5. Push public GitHub repo with Codex-attributed commits.
33
+ 6. Deploy Hugging Face Space after HF auth is available.
DEPLOYMENT.md ADDED
@@ -0,0 +1,65 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Deployment
2
+
3
+ ## GitHub
4
+
5
+ Public repo:
6
+
7
+ https://github.com/rishabhsai/offline-form-pilot
8
+
9
+ The commit history uses Codex-attributed commit messages for the OpenAI/Codex track.
10
+
11
+ ## Hugging Face Space
12
+
13
+ The local `hf` CLI is installed in the project venv. Authenticate first:
14
+
15
+ ```bash
16
+ source .venv/bin/activate
17
+ hf auth login
18
+ ```
19
+
20
+ If late org access is approved:
21
+
22
+ ```bash
23
+ hf repos create build-small-hackathon/offline-form-pilot --type space --space-sdk gradio --exist-ok
24
+ hf upload build-small-hackathon/offline-form-pilot . --type space \
25
+ --exclude ".venv/*" \
26
+ --exclude "traces/*" \
27
+ --exclude "exports/*" \
28
+ --commit-message "Codex: upload Offline Form Pilot Space"
29
+ ```
30
+
31
+ If org access is not available, use a personal Space:
32
+
33
+ ```bash
34
+ hf repos create rishabhsai/offline-form-pilot --type space --space-sdk gradio --exist-ok
35
+ hf upload rishabhsai/offline-form-pilot . --type space \
36
+ --exclude ".venv/*" \
37
+ --exclude "traces/*" \
38
+ --exclude "exports/*" \
39
+ --commit-message "Codex: upload Offline Form Pilot Space"
40
+ ```
41
+
42
+ Current personal Space:
43
+
44
+ https://huggingface.co/spaces/rishabhsai/offline-form-pilot
45
+
46
+ After deployment, update `README.md` with the Space URL and push one final Codex-attributed commit to GitHub.
47
+
48
+ ## Space Secrets
49
+
50
+ For the optional small-model assist mode, add one of these secrets in the Space settings:
51
+
52
+ - `HF_TOKEN`
53
+ - `HUGGINGFACEHUB_API_TOKEN`
54
+
55
+ The app still runs without a token using the local structured matcher.
56
+
57
+ ## Demo Checklist
58
+
59
+ 1. Paste a messy form.
60
+ 2. Paste safe sample facts.
61
+ 3. Run local structured matcher.
62
+ 4. Switch to small-model assist if the Space token is configured.
63
+ 5. Show ready/review/missing fields.
64
+ 6. Show generated questions before copying.
65
+ 7. Download trace JSON and field CSV.
FIELD_NOTES.md ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Field Notes
2
+
3
+ ## Question
4
+
5
+ Can a small model make form filling safer and less frustrating by turning messy paperwork into a reviewable field table instead of directly controlling a browser or submitting a form?
6
+
7
+ ## Intended User
8
+
9
+ One real person who regularly deals with confusing forms or paperwork. The project should name the workflow in the demo without exposing private data.
10
+
11
+ ## What The App Does
12
+
13
+ - Finds likely fields in pasted form text.
14
+ - Matches user-provided facts to those fields.
15
+ - Separates confident fills from missing or risky fields.
16
+ - Generates plain-English questions for the missing fields.
17
+ - Exports an anonymized trace for review.
18
+
19
+ ## What The App Does Not Do
20
+
21
+ - It does not submit forms.
22
+ - It does not bypass user review.
23
+ - It does not claim legal, medical, financial, or immigration advice.
24
+
25
+ ## Early Hypothesis
26
+
27
+ The useful small-model behavior is not "agent fills the website." The useful behavior is "agent prepares a clean checklist the person can verify quickly."
README.md CHANGED
@@ -1,13 +1,53 @@
1
  ---
2
  title: Offline Form Pilot
3
- emoji: 🐨
4
- colorFrom: pink
5
- colorTo: green
6
  sdk: gradio
7
  sdk_version: 6.16.0
8
- python_version: '3.13'
9
  app_file: app.py
10
  pinned: false
 
 
11
  ---
12
 
13
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  title: Offline Form Pilot
3
+ colorFrom: green
4
+ colorTo: blue
 
5
  sdk: gradio
6
  sdk_version: 6.16.0
 
7
  app_file: app.py
8
  pinned: false
9
+ license: mit
10
+ short_description: Review confusing forms before copying fields.
11
  ---
12
 
13
+ # Offline Form Pilot
14
+
15
+ Offline Form Pilot helps a real person fill confusing forms without handing control to an autopilot. Paste a form, paste the facts you are comfortable using, and the app produces a review table with proposed values, confidence, missing fields, and questions to ask before anything is copied.
16
+
17
+ The app is designed for the Build Small Hackathon:
18
+
19
+ - **Track:** Backyard AI.
20
+ - **Small model constraint:** default target is a <=4B small model.
21
+ - **Safety posture:** no automatic submission, no hidden form filling, no legal/financial certainty.
22
+ - **Codex track:** this public repo was built with Codex-attributed commits and includes `CODEX_BUILD_LOG.md`.
23
+
24
+ ## Why Small Models Fit
25
+
26
+ Form help is mostly narrow extraction, matching, and clarification. A small model can do useful work when the interface forces structured fields, review, and missing-information checks.
27
+
28
+ ## Local Run
29
+
30
+ ```bash
31
+ python3 -m venv .venv
32
+ source .venv/bin/activate
33
+ pip install -r requirements.txt
34
+ python app.py
35
+ ```
36
+
37
+ ## Deployment
38
+
39
+ See `DEPLOYMENT.md` for the GitHub and Hugging Face Space commands.
40
+
41
+ ## Submission Checklist
42
+
43
+ - Public GitHub repo linked from this README.
44
+ - Hugging Face Space link added after deployment.
45
+ - Short demo video.
46
+ - Social post.
47
+ - Field notes.
48
+ - Optional trace dataset with anonymized examples.
49
+
50
+ ## Repository Links
51
+
52
+ - GitHub: https://github.com/rishabhsai/offline-form-pilot
53
+ - Hugging Face Space: https://huggingface.co/spaces/rishabhsai/offline-form-pilot
app.py ADDED
@@ -0,0 +1,236 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from __future__ import annotations
2
+
3
+ import json
4
+ from pathlib import Path
5
+ from typing import Any
6
+
7
+ import gradio as gr
8
+ import pandas as pd
9
+
10
+ from formpilot.engine import analyze_form, export_trace, rows_to_csv
11
+ from formpilot.model_assist import DEFAULT_SMALL_MODEL, try_hf_model_assist
12
+
13
+
14
+ APP_DIR = Path(__file__).resolve().parent
15
+ TRACE_DIR = APP_DIR / "traces"
16
+ EXPORT_DIR = APP_DIR / "exports"
17
+ TRACE_DIR.mkdir(exist_ok=True)
18
+ EXPORT_DIR.mkdir(exist_ok=True)
19
+
20
+ SAMPLE_FORM = """Community Center Membership Form
21
+
22
+ Full name: ____________________
23
+ Email: ____________________
24
+ Phone: ____________________
25
+ Address: ____________________
26
+ Emergency contact: ____________________
27
+ Relationship: ____________________
28
+ Signature: ____________________
29
+ Date: ____________________
30
+ """
31
+
32
+ SAMPLE_FACTS = """Full name: Jordan Lee
33
+ Email: jordan.lee@example.com
34
+ Phone: 555-0137
35
+ Address: 42 Maple Street, Springfield, NY 10027
36
+ Emergency contact: Priya Lee
37
+ Relationship: Sister
38
+ Date: June 6, 2026
39
+ """
40
+
41
+ CSS = """
42
+ .gradio-container {
43
+ max-width: 1240px !important;
44
+ }
45
+ #hero {
46
+ padding: 8px 0 16px 0;
47
+ border-bottom: 1px solid #d7dde5;
48
+ }
49
+ #hero h1 {
50
+ font-size: 34px;
51
+ line-height: 1.05;
52
+ letter-spacing: 0;
53
+ margin: 0 0 8px 0;
54
+ }
55
+ #hero p {
56
+ color: #5b6470;
57
+ max-width: 860px;
58
+ font-size: 15px;
59
+ }
60
+ .status-ready {
61
+ color: #1f6f4a;
62
+ font-weight: 700;
63
+ }
64
+ .status-review {
65
+ color: #9a6500;
66
+ font-weight: 700;
67
+ }
68
+ .status-missing {
69
+ color: #a33d3d;
70
+ font-weight: 700;
71
+ }
72
+ .panel-note {
73
+ border: 1px solid #d7dde5;
74
+ border-radius: 8px;
75
+ background: #ffffff;
76
+ padding: 12px 14px;
77
+ }
78
+ """
79
+
80
+
81
+ def _status_html(rows: list[dict[str, Any]], risks: list[str]) -> str:
82
+ ready = sum(1 for row in rows if row["status"] == "ready")
83
+ review = sum(1 for row in rows if row["status"] == "review")
84
+ missing = sum(1 for row in rows if row["status"] == "missing")
85
+ risk_items = "".join(f"<li>{risk}</li>" for risk in risks)
86
+ return f"""
87
+ <div class="panel-note">
88
+ <p><span class="status-ready">{ready} ready</span> · <span class="status-review">{review} review</span> · <span class="status-missing">{missing} missing</span></p>
89
+ <ul>{risk_items}</ul>
90
+ </div>
91
+ """
92
+
93
+
94
+ def _write_export_files(payload: dict[str, Any]) -> tuple[str, str]:
95
+ trace_path = export_trace(payload, TRACE_DIR)
96
+ csv_path = EXPORT_DIR / "formpilot_latest_fields.csv"
97
+ csv_path.write_text(rows_to_csv(payload["rows"]), encoding="utf-8")
98
+ return trace_path, str(csv_path)
99
+
100
+
101
+ def _model_payload_to_rows(model_payload: dict[str, Any]) -> list[dict[str, Any]]:
102
+ rows = model_payload.get("fields", [])
103
+ normalized = []
104
+ for row in rows:
105
+ normalized.append(
106
+ {
107
+ "field": str(row.get("field", "")),
108
+ "proposed_value": str(row.get("proposed_value", "")),
109
+ "status": str(row.get("status", "review")),
110
+ "confidence": int(row.get("confidence", 0) or 0),
111
+ "source": str(row.get("source", "")),
112
+ "note": str(row.get("note", "")),
113
+ }
114
+ )
115
+ return normalized
116
+
117
+
118
+ def run_pilot(
119
+ form_text: str,
120
+ user_facts: str,
121
+ mode: str,
122
+ model_id: str,
123
+ ) -> tuple[pd.DataFrame, str, str, str, dict[str, Any], str, str]:
124
+ if not form_text.strip():
125
+ raise gr.Error("Paste a form or request first.")
126
+
127
+ payload = analyze_form(form_text, user_facts)
128
+ backend_note = "Local structured matcher"
129
+
130
+ if mode == "Small model assist":
131
+ try:
132
+ model_payload = try_hf_model_assist(form_text, user_facts, model_id.strip() or DEFAULT_SMALL_MODEL)
133
+ model_rows = _model_payload_to_rows(model_payload)
134
+ if model_rows:
135
+ payload["rows"] = model_rows
136
+ payload["questions"] = model_payload.get("questions", payload["questions"])
137
+ payload["risk_summary"] = model_payload.get("risk_summary", payload["risk_summary"])
138
+ payload["copy_ready"] = "\n".join(
139
+ f"{row['field']}: {row['proposed_value'] or '[NEEDS USER INPUT]'}"
140
+ for row in model_rows
141
+ )
142
+ backend_note = f"Small model assist: {model_id.strip() or DEFAULT_SMALL_MODEL}"
143
+ except Exception as exc:
144
+ payload["risk_summary"].insert(
145
+ 0,
146
+ f"Small model assist failed; used local matcher instead. Reason: {exc}",
147
+ )
148
+
149
+ payload["backend"] = backend_note
150
+ trace_path, csv_path = _write_export_files(payload)
151
+
152
+ rows = payload["rows"]
153
+ table = pd.DataFrame(rows, columns=["field", "proposed_value", "status", "confidence", "source", "note"])
154
+ questions = "\n".join(f"- {question}" for question in payload["questions"]) or "No missing-field questions detected."
155
+ summary = _status_html(rows, payload["risk_summary"])
156
+ return table, payload["copy_ready"], questions, summary, payload, trace_path, csv_path
157
+
158
+
159
+ def clear_outputs() -> tuple[pd.DataFrame, str, str, str, dict[str, Any], None, None]:
160
+ return pd.DataFrame(), "", "", "", {}, None, None
161
+
162
+
163
+ def build_demo() -> gr.Blocks:
164
+ with gr.Blocks(title="Offline Form Pilot") as demo:
165
+ gr.Markdown(
166
+ """
167
+ # Offline Form Pilot
168
+ Paste a form and the facts you are willing to use. The app prepares a review table, missing-field questions, and copy-ready text without submitting anything.
169
+ """,
170
+ elem_id="hero",
171
+ )
172
+
173
+ with gr.Row():
174
+ with gr.Column(scale=5):
175
+ form_text = gr.Textbox(
176
+ label="Form or request text",
177
+ value=SAMPLE_FORM,
178
+ lines=12,
179
+ max_lines=18,
180
+ )
181
+ user_facts = gr.Textbox(
182
+ label="User facts",
183
+ value=SAMPLE_FACTS,
184
+ lines=10,
185
+ max_lines=16,
186
+ )
187
+ with gr.Column(scale=3):
188
+ mode = gr.Radio(
189
+ label="Analysis mode",
190
+ choices=["Local structured matcher", "Small model assist"],
191
+ value="Local structured matcher",
192
+ )
193
+ model_id = gr.Textbox(label="Small model id", value=DEFAULT_SMALL_MODEL)
194
+ run_btn = gr.Button("Prepare form review", variant="primary")
195
+ clear_btn = gr.Button("Clear outputs")
196
+ gr.Markdown(
197
+ """
198
+ Human review is required. Do not paste secrets unless you are comfortable with the selected backend.
199
+ """,
200
+ elem_classes=["panel-note"],
201
+ )
202
+
203
+ summary = gr.HTML()
204
+ table = gr.Dataframe(
205
+ label="Review table",
206
+ headers=["field", "proposed_value", "status", "confidence", "source", "note"],
207
+ wrap=True,
208
+ interactive=False,
209
+ )
210
+ with gr.Row():
211
+ copy_ready = gr.Textbox(label="Copy-ready draft", lines=10, buttons=["copy"])
212
+ questions = gr.Textbox(label="Questions before copying", lines=10, buttons=["copy"])
213
+ with gr.Accordion("Trace and exports", open=False):
214
+ raw_json = gr.JSON(label="Trace JSON")
215
+ trace_file = gr.File(label="Download trace JSON")
216
+ csv_file = gr.File(label="Download field CSV")
217
+
218
+ run_btn.click(
219
+ run_pilot,
220
+ inputs=[form_text, user_facts, mode, model_id],
221
+ outputs=[table, copy_ready, questions, summary, raw_json, trace_file, csv_file],
222
+ api_name="prepare_form_review",
223
+ )
224
+ clear_btn.click(
225
+ clear_outputs,
226
+ outputs=[table, copy_ready, questions, summary, raw_json, trace_file, csv_file],
227
+ )
228
+
229
+ return demo
230
+
231
+
232
+ demo = build_demo()
233
+
234
+
235
+ if __name__ == "__main__":
236
+ demo.launch(css=CSS)
examples/anonymized_trace.jsonl ADDED
@@ -0,0 +1 @@
 
 
1
+ {"case_id":"community_center_demo","input_type":"pasted_text","form_fields":["Full name","Email","Phone","Address","Emergency contact","Relationship","Signature","Date"],"available_facts":["full name","email","phone","address","emergency contact","relationship","date"],"outcome":{"ready":7,"review":0,"missing":1},"missing_fields":["Signature"],"note":"Synthetic trace for demo and Sharing is Caring badge planning."}
formpilot/__init__.py ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ """Offline Form Pilot package."""
2
+
3
+ from formpilot.engine import analyze_form
4
+
5
+ __all__ = ["analyze_form"]
formpilot/engine.py ADDED
@@ -0,0 +1,328 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Structured form analysis for Offline Form Pilot."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import csv
6
+ import io
7
+ import json
8
+ import re
9
+ from dataclasses import asdict, dataclass
10
+ from datetime import datetime, timezone
11
+ from difflib import SequenceMatcher
12
+ from pathlib import Path
13
+ from typing import Any
14
+
15
+
16
+ COMMON_FIELDS = [
17
+ "full name",
18
+ "first name",
19
+ "last name",
20
+ "date of birth",
21
+ "email",
22
+ "phone",
23
+ "address",
24
+ "city",
25
+ "state",
26
+ "postal code",
27
+ "zip code",
28
+ "country",
29
+ "employer",
30
+ "school",
31
+ "student id",
32
+ "account number",
33
+ "policy number",
34
+ "emergency contact",
35
+ "relationship",
36
+ "signature",
37
+ "date",
38
+ ]
39
+
40
+ SENSITIVE_TERMS = {
41
+ "ssn",
42
+ "social security",
43
+ "passport",
44
+ "bank",
45
+ "routing",
46
+ "account",
47
+ "card",
48
+ "credit",
49
+ "medical",
50
+ "diagnosis",
51
+ "tax",
52
+ "visa",
53
+ "immigration",
54
+ }
55
+
56
+ STOPWORDS = {
57
+ "the",
58
+ "a",
59
+ "an",
60
+ "your",
61
+ "you",
62
+ "of",
63
+ "for",
64
+ "and",
65
+ "or",
66
+ "to",
67
+ "in",
68
+ "on",
69
+ "with",
70
+ "please",
71
+ "enter",
72
+ "provide",
73
+ }
74
+
75
+
76
+ @dataclass(frozen=True)
77
+ class FieldMatch:
78
+ field: str
79
+ proposed_value: str
80
+ status: str
81
+ confidence: int
82
+ source: str
83
+ note: str
84
+
85
+
86
+ def normalize_label(text: str) -> str:
87
+ """Normalize labels for fuzzy matching."""
88
+
89
+ text = text.lower()
90
+ text = re.sub(r"[^a-z0-9]+", " ", text)
91
+ text = re.sub(r"\s+", " ", text).strip()
92
+ return text
93
+
94
+
95
+ def _tokens(text: str) -> set[str]:
96
+ return {token for token in normalize_label(text).split() if token not in STOPWORDS}
97
+
98
+
99
+ def parse_user_facts(raw_facts: str) -> dict[str, str]:
100
+ """Parse key-value facts from pasted user notes."""
101
+
102
+ facts: dict[str, str] = {}
103
+ free_lines: list[str] = []
104
+ for line in raw_facts.splitlines():
105
+ cleaned = line.strip().strip("-*")
106
+ if not cleaned:
107
+ continue
108
+ match = re.match(r"^([^:=]{2,60})\s*[:=]\s*(.+)$", cleaned)
109
+ if match:
110
+ key = normalize_label(match.group(1))
111
+ value = match.group(2).strip()
112
+ facts[key] = value
113
+ else:
114
+ free_lines.append(cleaned)
115
+
116
+ inferred = infer_facts_from_free_text("\n".join(free_lines))
117
+ for key, value in inferred.items():
118
+ facts.setdefault(key, value)
119
+ return facts
120
+
121
+
122
+ def infer_facts_from_free_text(text: str) -> dict[str, str]:
123
+ """Extract a small set of common facts from unstructured text."""
124
+
125
+ facts: dict[str, str] = {}
126
+ email = re.search(r"[\w.+-]+@[\w.-]+\.[a-zA-Z]{2,}", text)
127
+ if email:
128
+ facts["email"] = email.group(0)
129
+
130
+ phone = re.search(r"(?:\+?\d[\d .()-]{7,}\d)", text)
131
+ if phone:
132
+ facts["phone"] = phone.group(0).strip()
133
+
134
+ zip_code = re.search(r"\b\d{5}(?:-\d{4})?\b", text)
135
+ if zip_code:
136
+ facts["zip code"] = zip_code.group(0)
137
+ facts["postal code"] = zip_code.group(0)
138
+
139
+ dob = re.search(r"\b(?:dob|date of birth)\s*[:=]?\s*([A-Za-z0-9, /.-]{6,20})", text, re.I)
140
+ if dob:
141
+ facts["date of birth"] = dob.group(1).strip()
142
+
143
+ return facts
144
+
145
+
146
+ def detect_fields(form_text: str) -> list[str]:
147
+ """Find likely form fields from pasted form text."""
148
+
149
+ candidates: list[str] = []
150
+
151
+ for line in form_text.splitlines():
152
+ cleaned = line.strip()
153
+ if not cleaned:
154
+ continue
155
+ cleaned = re.sub(r"\s+", " ", cleaned)
156
+
157
+ label_match = re.match(r"^([A-Za-z][A-Za-z0-9 /'().,-]{1,70})\s*[:_]{1,}\s*(?:\[\s*\])?\s*$", cleaned)
158
+ if label_match:
159
+ candidates.append(label_match.group(1))
160
+ continue
161
+
162
+ bracket_match = re.match(r"^([A-Za-z][A-Za-z0-9 /'().,-]{1,70})\s*\[\s*\]\s*$", cleaned)
163
+ if bracket_match:
164
+ candidates.append(bracket_match.group(1))
165
+ continue
166
+
167
+ inline_match = re.match(r"^([A-Za-z][A-Za-z0-9 /'().,-]{1,45})\s*:\s+_{2,}", cleaned)
168
+ if inline_match:
169
+ candidates.append(inline_match.group(1))
170
+
171
+ lowered_form = normalize_label(form_text)
172
+ for common in COMMON_FIELDS:
173
+ if common in lowered_form:
174
+ candidates.append(common)
175
+
176
+ unique: list[str] = []
177
+ seen: set[str] = set()
178
+ for candidate in candidates:
179
+ label = normalize_label(candidate)
180
+ if len(label) < 2 or label in seen:
181
+ continue
182
+ seen.add(label)
183
+ unique.append(candidate.strip(" :_"))
184
+ return unique
185
+
186
+
187
+ def match_field(field: str, facts: dict[str, str]) -> FieldMatch:
188
+ """Match one field label to available user facts."""
189
+
190
+ field_norm = normalize_label(field)
191
+ field_tokens = _tokens(field)
192
+ best_key = ""
193
+ best_score = 0.0
194
+
195
+ for key in facts:
196
+ key_tokens = _tokens(key)
197
+ overlap = len(field_tokens & key_tokens) / max(1, len(field_tokens | key_tokens))
198
+ ratio = SequenceMatcher(None, field_norm, key).ratio()
199
+ score = max(overlap, ratio * 0.85)
200
+ if score > best_score:
201
+ best_key = key
202
+ best_score = score
203
+
204
+ if not best_key:
205
+ return _missing_match(field, "No matching user fact found.")
206
+
207
+ value = facts[best_key]
208
+ sensitive = is_sensitive_field(field)
209
+ if best_score >= 0.86 or field_norm == best_key:
210
+ confidence = 95 if not sensitive else 84
211
+ status = "review" if sensitive else "ready"
212
+ note = "Strong label match."
213
+ elif best_score >= 0.58:
214
+ confidence = int(best_score * 100)
215
+ status = "review"
216
+ note = f"Possible match from '{best_key}'."
217
+ else:
218
+ return _missing_match(field, "No close enough user fact found.")
219
+
220
+ if sensitive:
221
+ note += " Sensitive field: verify manually before copying."
222
+
223
+ return FieldMatch(
224
+ field=field,
225
+ proposed_value=value,
226
+ status=status,
227
+ confidence=confidence,
228
+ source=best_key,
229
+ note=note,
230
+ )
231
+
232
+
233
+ def _missing_match(field: str, note: str) -> FieldMatch:
234
+ return FieldMatch(
235
+ field=field,
236
+ proposed_value="",
237
+ status="missing",
238
+ confidence=0,
239
+ source="",
240
+ note=note,
241
+ )
242
+
243
+
244
+ def is_sensitive_field(field: str) -> bool:
245
+ label = normalize_label(field)
246
+ return any(term in label for term in SENSITIVE_TERMS)
247
+
248
+
249
+ def questions_for_missing(matches: list[FieldMatch]) -> list[str]:
250
+ """Generate plain-English follow-up questions for missing fields."""
251
+
252
+ questions = []
253
+ for match in matches:
254
+ if match.status == "missing":
255
+ questions.append(f"What should go in '{match.field}'?")
256
+ return questions
257
+
258
+
259
+ def _risk_summary(matches: list[FieldMatch]) -> list[str]:
260
+ risks = []
261
+ missing = sum(1 for match in matches if match.status == "missing")
262
+ review = sum(1 for match in matches if match.status == "review")
263
+ sensitive = sum(1 for match in matches if is_sensitive_field(match.field))
264
+ if missing:
265
+ risks.append(f"{missing} field(s) still need information.")
266
+ if review:
267
+ risks.append(f"{review} field(s) should be reviewed before copying.")
268
+ if sensitive:
269
+ risks.append(f"{sensitive} sensitive field(s) detected.")
270
+ if not risks:
271
+ risks.append("All detected fields have proposed values, but user review is still required.")
272
+ return risks
273
+
274
+
275
+ def analyze_form(form_text: str, user_facts: str, use_demo_fields: bool = True) -> dict[str, Any]:
276
+ """Analyze form text and facts into reviewable outputs."""
277
+
278
+ fields = detect_fields(form_text)
279
+ if not fields and use_demo_fields:
280
+ fields = ["Full name", "Email", "Phone", "Address", "Date", "Signature"]
281
+
282
+ facts = parse_user_facts(user_facts)
283
+ matches = [match_field(field, facts) for field in fields]
284
+ rows = [asdict(match) for match in matches]
285
+ return {
286
+ "created_at": datetime.now(timezone.utc).isoformat(),
287
+ "fields": fields,
288
+ "facts": facts,
289
+ "rows": rows,
290
+ "questions": questions_for_missing(matches),
291
+ "risk_summary": _risk_summary(matches),
292
+ "copy_ready": copy_ready_text(matches),
293
+ }
294
+
295
+
296
+ def copy_ready_text(matches: list[FieldMatch]) -> str:
297
+ """Create a conservative copy-ready field list."""
298
+
299
+ lines = []
300
+ for match in matches:
301
+ value = match.proposed_value if match.proposed_value else "[NEEDS USER INPUT]"
302
+ flag = " REVIEW" if match.status == "review" else ""
303
+ lines.append(f"{match.field}: {value}{flag}")
304
+ return "\n".join(lines)
305
+
306
+
307
+ def export_trace(payload: dict[str, Any], directory: Path | str = "traces") -> str:
308
+ """Write one anonymizable JSON trace and return its path."""
309
+
310
+ out_dir = Path(directory)
311
+ out_dir.mkdir(parents=True, exist_ok=True)
312
+ stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
313
+ path = out_dir / f"formpilot_trace_{stamp}.json"
314
+ path.write_text(json.dumps(payload, indent=2), encoding="utf-8")
315
+ return str(path)
316
+
317
+
318
+ def rows_to_csv(rows: list[dict[str, Any]]) -> str:
319
+ """Serialize rows for quick export."""
320
+
321
+ output = io.StringIO()
322
+ writer = csv.DictWriter(
323
+ output,
324
+ fieldnames=["field", "proposed_value", "status", "confidence", "source", "note"],
325
+ )
326
+ writer.writeheader()
327
+ writer.writerows(rows)
328
+ return output.getvalue()
formpilot/model_assist.py ADDED
@@ -0,0 +1,73 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Optional small-model assist for form analysis."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import json
6
+ import os
7
+ from typing import Any
8
+
9
+
10
+ DEFAULT_SMALL_MODEL = os.getenv("FORMPILOT_MODEL", "openbmb/MiniCPM5-1B")
11
+
12
+
13
+ def build_model_prompt(form_text: str, user_facts: str) -> str:
14
+ """Prompt a small model to return conservative form-fill JSON."""
15
+
16
+ return f"""You are helping prepare a form for human review. Do not submit anything.
17
+
18
+ Return only JSON with this schema:
19
+ {{
20
+ "fields": [
21
+ {{
22
+ "field": "field label",
23
+ "proposed_value": "value or empty string",
24
+ "status": "ready|review|missing",
25
+ "confidence": 0,
26
+ "source": "fact used or empty",
27
+ "note": "short reason"
28
+ }}
29
+ ],
30
+ "questions": ["questions for missing fields"],
31
+ "risk_summary": ["review warnings"]
32
+ }}
33
+
34
+ Rules:
35
+ - Use only the user facts.
36
+ - If a value is absent, mark missing.
37
+ - Sensitive fields must be review, not ready.
38
+ - Never invent account numbers, IDs, dates, signatures, addresses, or legal facts.
39
+
40
+ FORM:
41
+ {form_text}
42
+
43
+ USER FACTS:
44
+ {user_facts}
45
+ """
46
+
47
+
48
+ def try_hf_model_assist(form_text: str, user_facts: str, model_id: str = DEFAULT_SMALL_MODEL) -> dict[str, Any]:
49
+ """Call a small Hugging Face model and parse its JSON response."""
50
+
51
+ try:
52
+ from huggingface_hub import InferenceClient
53
+ except ImportError as exc:
54
+ raise RuntimeError("huggingface_hub is not installed.") from exc
55
+
56
+ token = os.getenv("HF_TOKEN") or os.getenv("HUGGINGFACEHUB_API_TOKEN")
57
+ client = InferenceClient(model=model_id, token=token)
58
+ prompt = build_model_prompt(form_text, user_facts)
59
+ response = client.text_generation(
60
+ prompt,
61
+ max_new_tokens=700,
62
+ temperature=0.1,
63
+ return_full_text=False,
64
+ )
65
+ return _parse_json_response(str(response))
66
+
67
+
68
+ def _parse_json_response(raw: str) -> dict[str, Any]:
69
+ start = raw.find("{")
70
+ end = raw.rfind("}")
71
+ if start == -1 or end == -1 or end <= start:
72
+ raise ValueError("Model did not return a JSON object.")
73
+ return json.loads(raw[start : end + 1])
requirements.txt ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ gradio>=6.16.0
2
+ pandas>=2.2.0
3
+ pydantic>=2.0.0
4
+ huggingface_hub>=0.32.0
5
+ pytest>=8.0.0
tests/test_engine.py ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from formpilot.engine import analyze_form, detect_fields, parse_user_facts
2
+
3
+
4
+ def test_detect_fields_from_underlined_form() -> None:
5
+ form = """
6
+ Full name: ____________
7
+ Email: ________________
8
+ Emergency contact: ____
9
+ """
10
+ assert detect_fields(form) == ["Full name", "Email", "Emergency contact"]
11
+
12
+
13
+ def test_parse_user_facts_key_values_and_free_text() -> None:
14
+ facts = parse_user_facts(
15
+ """
16
+ Name: Jordan Lee
17
+ jordan.lee@example.com
18
+ Phone: 555-0137
19
+ """
20
+ )
21
+ assert facts["name"] == "Jordan Lee"
22
+ assert facts["email"] == "jordan.lee@example.com"
23
+ assert facts["phone"] == "555-0137"
24
+
25
+
26
+ def test_analyze_form_marks_missing_fields() -> None:
27
+ payload = analyze_form("Full name: ____\nEmail: ____\nSignature: ____", "Full name: Jordan Lee")
28
+ rows = {row["field"]: row for row in payload["rows"]}
29
+ assert rows["Full name"]["status"] == "ready"
30
+ assert rows["Email"]["status"] == "missing"
31
+ assert rows["Signature"]["status"] == "missing"
32
+ assert payload["questions"]