Update README.md
Browse files
README.md
CHANGED
|
@@ -13,7 +13,7 @@ short_description: A 1.5B agent that writes, runs and repairs its own EDA code
|
|
| 13 |
|
| 14 |
Point it at any Hugging Face dataset. It extracts the schema, writes pandas /
|
| 15 |
matplotlib analysis code, executes that code, repairs it from the traceback if it
|
| 16 |
-
crashes, and assembles a Markdown + PDF report with tables and
|
| 17 |
|
| 18 |
Built on **Qwen2.5-Coder-1.5B-Instruct**, small enough to run on a free GPU.
|
| 19 |
|
|
@@ -27,7 +27,7 @@ model, so the app is usable even when a visitor's daily GPU quota is exhausted.
|
|
| 27 |
| 1. Load + normalize | Python | `load_dataset` with a raw-file fallback; quoted nulls converted to real `NaN`; index-like columns dropped |
|
| 28 |
| 2. Context card | Python | One line per column: dtype, null count, cardinality, range |
|
| 29 |
| 3. Code generation | **LLM** | Schema + instruction β one fenced Python block |
|
| 30 |
-
| 4. Execution | Python | `exec` in an isolated namespace, stdout captured,
|
| 31 |
| 5. Repair (Γ1) | **LLM** | Cleaned traceback (offending line + message) β corrected script |
|
| 32 |
| 6. Facts + tables | Python | `idxmax`, `describe`, `corr` β Markdown tables |
|
| 33 |
| 7. Narrative | **LLM** | One sentence per pre-computed fact |
|
|
@@ -35,7 +35,7 @@ model, so the app is usable even when a visitor's daily GPU quota is exhausted.
|
|
| 35 |
## Design note
|
| 36 |
|
| 37 |
The model owns two jobs: writing analysis code and repairing it. Everything else β
|
| 38 |
-
table rendering,
|
| 39 |
in Python and handed to the model as a stated fact.
|
| 40 |
|
| 41 |
This was arrived at empirically. A 1.5B model reliably produced correct *descriptions*
|
|
@@ -50,7 +50,7 @@ instruction to it.
|
|
| 50 |
repair loop: it failed on both test datasets, and the model fixed it correctly once
|
| 51 |
(`select_dtypes(...).corr()`) and incorrectly once (`dropna(subset=[...]).corr()`).
|
| 52 |
|
| 53 |
-
##
|
| 54 |
|
| 55 |
Generated plotting code is wrapped rather than trusted. `seaborn.heatmap`,
|
| 56 |
`seaborn.histplot` and `plt.savefig` are patched around execution so that, on any
|
|
@@ -69,7 +69,7 @@ section, since they are almost always duplicate encodings rather than findings.
|
|
| 69 |
|
| 70 |
## Known limitations
|
| 71 |
|
| 72 |
-
- The model is text-only and never sees the
|
| 73 |
prevented from describing them.
|
| 74 |
- `audit_numbers` flags numerals absent from the evidence. It detects fabrication,
|
| 75 |
not misinterpretation, and false-positives on derived values.
|
|
|
|
| 13 |
|
| 14 |
Point it at any Hugging Face dataset. It extracts the schema, writes pandas /
|
| 15 |
matplotlib analysis code, executes that code, repairs it from the traceback if it
|
| 16 |
+
crashes, and assembles a Markdown + PDF report with tables and plots.
|
| 17 |
|
| 18 |
Built on **Qwen2.5-Coder-1.5B-Instruct**, small enough to run on a free GPU.
|
| 19 |
|
|
|
|
| 27 |
| 1. Load + normalize | Python | `load_dataset` with a raw-file fallback; quoted nulls converted to real `NaN`; index-like columns dropped |
|
| 28 |
| 2. Context card | Python | One line per column: dtype, null count, cardinality, range |
|
| 29 |
| 3. Code generation | **LLM** | Schema + instruction β one fenced Python block |
|
| 30 |
+
| 4. Execution | Python | `exec` in an isolated namespace, stdout captured, plots collected |
|
| 31 |
| 5. Repair (Γ1) | **LLM** | Cleaned traceback (offending line + message) β corrected script |
|
| 32 |
| 6. Facts + tables | Python | `idxmax`, `describe`, `corr` β Markdown tables |
|
| 33 |
| 7. Narrative | **LLM** | One sentence per pre-computed fact |
|
|
|
|
| 35 |
## Design note
|
| 36 |
|
| 37 |
The model owns two jobs: writing analysis code and repairing it. Everything else β
|
| 38 |
+
table rendering, plot placement, and every "which is highest" lookup β is computed
|
| 39 |
in Python and handed to the model as a stated fact.
|
| 40 |
|
| 41 |
This was arrived at empirically. A 1.5B model reliably produced correct *descriptions*
|
|
|
|
| 50 |
repair loop: it failed on both test datasets, and the model fixed it correctly once
|
| 51 |
(`select_dtypes(...).corr()`) and incorrectly once (`dropna(subset=[...]).corr()`).
|
| 52 |
|
| 53 |
+
## Plot and table quality
|
| 54 |
|
| 55 |
Generated plotting code is wrapped rather than trusted. `seaborn.heatmap`,
|
| 56 |
`seaborn.histplot` and `plt.savefig` are patched around execution so that, on any
|
|
|
|
| 69 |
|
| 70 |
## Known limitations
|
| 71 |
|
| 72 |
+
- The model is text-only and never sees the plots it generates; it is explicitly
|
| 73 |
prevented from describing them.
|
| 74 |
- `audit_numbers` flags numerals absent from the evidence. It detects fabrication,
|
| 75 |
not misinterpretation, and false-positives on derived values.
|