Update README.md
Browse files
README.md
CHANGED
|
@@ -9,13 +9,16 @@ pinned: false
|
|
| 9 |
short_description: A 1.5B agent that writes, runs and repairs its own EDA code
|
| 10 |
---
|
| 11 |
|
| 12 |
-
# EDA Agent
|
| 13 |
|
| 14 |
-
|
|
|
|
|
|
|
| 15 |
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
|
|
|
| 19 |
|
| 20 |
## Architecture
|
| 21 |
|
|
@@ -32,49 +35,74 @@ traceback if it crashes, and assembles a Markdown report with tables and figures
|
|
| 32 |
## Design note
|
| 33 |
|
| 34 |
The model owns two jobs: writing analysis code and repairing it. Everything else β
|
| 35 |
-
table rendering, figure placement, and every "which is highest" lookup β is computed
|
| 36 |
-
Python and handed to the model as a stated fact.
|
| 37 |
|
| 38 |
This was arrived at empirically. A 1.5B model reliably produced correct *descriptions*
|
| 39 |
and unreliable *lookups*: it rotated group means against their labels, invented
|
| 40 |
-
correlation pairs from fragments of two real ones, and inverted orderings. Adding
|
| 41 |
-
to the prompt made compliance worse, not better β at six simultaneous
|
| 42 |
-
stopped writing prose and began transcribing the input's structure.
|
| 43 |
-
worked removed responsibility from the model rather than adding
|
|
|
|
| 44 |
|
| 45 |
`df.corr()` on mixed-type data is patched deterministically rather than left to the
|
| 46 |
repair loop: it failed on both test datasets, and the model fixed it correctly once
|
| 47 |
(`select_dtypes(...).corr()`) and incorrectly once (`dropna(subset=[...]).corr()`).
|
| 48 |
|
| 49 |
-
##
|
| 50 |
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
|
| 58 |
## Known limitations
|
| 59 |
|
| 60 |
- The model is text-only and never sees the figures it generates; it is explicitly
|
| 61 |
prevented from describing them.
|
| 62 |
-
- `audit_numbers` flags numerals absent from the evidence. It detects fabrication,
|
| 63 |
-
misinterpretation, and false-positives on derived values.
|
| 64 |
-
- The model occasionally embellishes a supplied fact
|
| 65 |
-
|
| 66 |
-
every lookup has been moved into Python.
|
| 67 |
- Non-tabular, image and audio datasets are not supported.
|
| 68 |
- One repair attempt only. If it fails, the report is built from the data alone.
|
| 69 |
|
| 70 |
## Security
|
| 71 |
|
| 72 |
-
This Space executes LLM-generated Python. A static check blocks `subprocess`,
|
| 73 |
-
`open()`, `eval`/`exec` and `os`/`sys` imports, but `exec` is not a
|
| 74 |
-
guard as a speed bump. **Do not add secrets to this Space's
|
|
|
|
| 75 |
|
| 76 |
## Hardware
|
| 77 |
|
| 78 |
Built for **ZeroGPU**. The model is placed on `cuda` at module level and all GPU work
|
| 79 |
-
runs inside a `@spaces.GPU` function
|
| 80 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 9 |
short_description: A 1.5B agent that writes, runs and repairs its own EDA code
|
| 10 |
---
|
| 11 |
|
| 12 |
+
# π EDA Agent
|
| 13 |
|
| 14 |
+
Point it at any Hugging Face dataset. It extracts the schema, writes pandas /
|
| 15 |
+
matplotlib analysis code, executes that code, repairs it from the traceback if it
|
| 16 |
+
crashes, and assembles a Markdown + PDF report with tables and figures.
|
| 17 |
|
| 18 |
+
Built on **Qwen2.5-Coder-1.5B-Instruct**, small enough to run on a free GPU.
|
| 19 |
+
|
| 20 |
+
Three **Quick Starters** are pre-generated and load instantly without touching the
|
| 21 |
+
model, so the app is usable even when a visitor's daily GPU quota is exhausted.
|
| 22 |
|
| 23 |
## Architecture
|
| 24 |
|
|
|
|
| 35 |
## Design note
|
| 36 |
|
| 37 |
The model owns two jobs: writing analysis code and repairing it. Everything else β
|
| 38 |
+
table rendering, figure placement, and every "which is highest" lookup β is computed
|
| 39 |
+
in Python and handed to the model as a stated fact.
|
| 40 |
|
| 41 |
This was arrived at empirically. A 1.5B model reliably produced correct *descriptions*
|
| 42 |
and unreliable *lookups*: it rotated group means against their labels, invented
|
| 43 |
+
correlation pairs from fragments of two real ones, and inverted orderings. Adding
|
| 44 |
+
rules to the prompt made compliance worse, not better β at six simultaneous
|
| 45 |
+
constraints it stopped writing prose and began transcribing the input's structure.
|
| 46 |
+
Every fix that worked removed responsibility from the model rather than adding
|
| 47 |
+
instruction to it.
|
| 48 |
|
| 49 |
`df.corr()` on mixed-type data is patched deterministically rather than left to the
|
| 50 |
repair loop: it failed on both test datasets, and the model fixed it correctly once
|
| 51 |
(`select_dtypes(...).corr()`) and incorrectly once (`dropna(subset=[...]).corr()`).
|
| 52 |
|
| 53 |
+
## Figure and table quality
|
| 54 |
|
| 55 |
+
Generated plotting code is wrapped rather than trusted. `seaborn.heatmap`,
|
| 56 |
+
`seaborn.histplot` and `plt.savefig` are patched around execution so that, on any
|
| 57 |
+
dataset:
|
| 58 |
+
|
| 59 |
+
- correlation heatmaps scale their canvas with the matrix and drop cell annotations
|
| 60 |
+
past 12 columns;
|
| 61 |
+
- long column names are abbreviated while keeping their suffixes;
|
| 62 |
+
- axes with more than 20 tick labels are thinned;
|
| 63 |
+
- heavily right-skewed distributions (prices, fares, incomes) are replotted on a log
|
| 64 |
+
axis;
|
| 65 |
+
- no axis or table ever shows scientific notation β `37,000,000`, not `3.7e+07`.
|
| 66 |
+
|
| 67 |
+
Near-perfect correlations (|r| β₯ 0.99) are separated into a **Redundant Columns**
|
| 68 |
+
section, since they are almost always duplicate encodings rather than findings.
|
| 69 |
|
| 70 |
## Known limitations
|
| 71 |
|
| 72 |
- The model is text-only and never sees the figures it generates; it is explicitly
|
| 73 |
prevented from describing them.
|
| 74 |
+
- `audit_numbers` flags numerals absent from the evidence. It detects fabrication,
|
| 75 |
+
not misinterpretation, and false-positives on derived values.
|
| 76 |
+
- The model occasionally embellishes a supplied fact. This is the residual failure
|
| 77 |
+
mode once every lookup has been moved into Python.
|
|
|
|
| 78 |
- Non-tabular, image and audio datasets are not supported.
|
| 79 |
- One repair attempt only. If it fails, the report is built from the data alone.
|
| 80 |
|
| 81 |
## Security
|
| 82 |
|
| 83 |
+
This Space executes LLM-generated Python. A static check blocks `subprocess`,
|
| 84 |
+
`socket`, `open()`, `eval`/`exec` and `os`/`sys` imports, but `exec` is not a
|
| 85 |
+
sandbox β treat the guard as a speed bump. **Do not add secrets to this Space's
|
| 86 |
+
settings.**
|
| 87 |
|
| 88 |
## Hardware
|
| 89 |
|
| 90 |
Built for **ZeroGPU**. The model is placed on `cuda` at module level and all GPU work
|
| 91 |
+
runs inside a `@spaces.GPU` function; the app also falls back to CPU so it boots
|
| 92 |
+
anywhere. One fresh report is about seven generation calls, roughly 60β90 s of GPU
|
| 93 |
+
time. Results are cached in memory, and the Quick Starters are served from disk, so
|
| 94 |
+
repeat views cost nothing.
|
| 95 |
+
|
| 96 |
+
## Repository layout
|
| 97 |
+
|
| 98 |
+
```
|
| 99 |
+
app.py the agent and the Gradio UI
|
| 100 |
+
requirements.txt
|
| 101 |
+
quickstarts/
|
| 102 |
+
titanic/ meta.json report.md code.py stdout.txt report.pdf plots/*.png
|
| 103 |
+
iris/ ...
|
| 104 |
+
housing/ ...
|
| 105 |
+
```
|
| 106 |
+
|
| 107 |
+
`quickstarts/` is generated by the companion Colab notebook. If it is absent, the
|
| 108 |
+
Quick Starter buttons fall back to a live run.
|