mcp-surface-lint / README.md
robworks-software's picture
Correct the upstream finding write-up and link gradio-app/gradio#13670
f409946 verified
|
Raw
History Blame Contribute Delete
4.24 kB
---
title: MCP Surface Lint
emoji: 🔎
colorFrom: indigo
colorTo: gray
sdk: gradio
sdk_version: 6.20.0
app_file: app.py
pinned: false
license: mit
short_description: Find where an MCP tool surface lets a caller fail silently
tags:
- mcp
- mcp-server
- developer-tools
- static-analysis
---
# MCP Surface Lint
Static checks over an MCP tool surface. Every rule answers one question:
> Could a caller do the wrong thing and get no error back?
A tool that crashes on bad input is fine. A tool that *accepts* bad input and
returns something plausible is the expensive kind of broken, because neither the
caller nor the person reading the transcript has any signal that it happened.
Paste a `tools/list` response, get back the specific places that can happen.
**This app is also an MCP server**, so an agent can lint an agent interface
without going through the UI.
## Using it as an MCP server
Launched with `demo.launch(mcp_server=True)`. Point a client at:
```
https://<space-host>/gradio_api/mcp/sse
```
Two tools are published:
| Tool | Purpose |
|---|---|
| `lint_tool_surface(surface_json)` | Run every rule over a surface, return findings with severities |
| `list_rules()` | Enumerate the checks, their ids, and their severities |
The UI event handlers are deliberately excluded with `api_name=False`. Without
that, Gradio also publishes `analyse`, `analyse_1`, and `explain` as tools: a
duplicate-name pair plus a handler taking a `gr.State` that cannot survive a tool
call. This linter flags all three patterns, so shipping them would be a poor look.
## The rules
| Id | Severity | Catches |
|---|---|---|
| MCP001 | high | Tool has no description |
| MCP002 | low | Description too short to distinguish from a neighbour |
| MCP003 | high | No `inputSchema` at all |
| MCP004 | high | Parameter with no description |
| MCP005 | high | Parameter with no declared type |
| MCP006 | medium | String parameter that is really a closed set, with no `enum` |
| MCP007 | high | Properties declared but no `required` list |
| MCP008 | low | Numeric parameter with no bounds |
| MCP009 | medium | Unconstrained object parameter |
| MCP010 | medium | Description never says what failure looks like |
| MCP011 | high | Destructive verb with no guard parameter and no warning |
| MCP012 | medium | Two tools a caller cannot choose between by name |
| MCP013 | medium | Two tools sharing an identical description |
Names are compared after crude stemming, because `list_user_records` and
`list_users_record` are the same tool as far as a caller is concerned and raw
token comparison misses that entirely.
## What it found in its own surface
Pointed at its own published schema, the linter returns exactly one finding, and
it is not in this repo:
```
[high] MCP007 lint_tool_surface: Schema declares properties but no 'required' list
```
Gradio's `get_input_schema` builds `{"type": "object", "properties": {...}}` and
never emits a `required` array. The information is already there: the same
parameter dicts it iterates carry `parameter_has_default`, and Gradio's own
`gradio skills` command uses that field to mark a parameter required. The
practical effect is that **every** Gradio MCP server publishes tools whose
arguments all look optional, so a caller that omits a mandatory one produces a
schema-valid call.
Reported upstream as [gradio-app/gradio#13670](https://github.com/gradio-app/gradio/issues/13670).
## Running locally
```bash
pip install -r requirements.txt
python app.py
```
Tests:
```bash
python -m pytest -q
```
Rules are only worth what their tests are worth, so there is also a mutation
matrix. It neuters each rule in turn and records which tests fail. A rule with no
failing test is either dead code or unguarded, and you cannot tell which without
looking:
```bash
python mutation_check.py
```
It refuses to report anything unless the baseline suite is green first, and it
disables bytecode caching, because every mutated file is the same length and
CPython validates a cached `.pyc` on `(mtime, size)`. Inside one second that pair
does not change, so the interpreter happily re-imports the previous mutation's
bytecode and the whole matrix reports the wrong rule's result.
## License
MIT.