# Adding an environment format Every environment the explorer shows is read by an **adapter** (`app/envs/contract.py`): Harbor task folders (`harbor.py`), Xiaomi's MiMo release (`mimo.py`), any dataset of rows (`rows.py`, with its row readers). An adapter answers the same five questions for its format, and the pages, the runner, rollouts and MCP then work for it with nothing else changed: | part | what the adapter gives | where it shows | |---|---|---| | tasks | `summary`, `tasks` (cards), `task` (one in full, as sections of blocks), `file`, `folder`, `random` | `/d//`, `/t///` | | environment | a section of the task: what it runs in | the task page | | harness | `run_options`: how it can run, by which runner, with which inputs; `materialize` / `run_task` for the runner | the run panel, `POST /api/runs` | | rollouts | `aliases`: the same task elsewhere (a raw row and its Harbor conversion); their rollouts are this task's | the task page, compare, community | | mcp | nothing to do: describe, list, search, read and files come from the above; `mcp_tools` / `mcp_call` add more | `/mcp/d//` | A task is named by a **ref**, a string the adapter defines and parses: a Harbor task's folder (`tasks/fix-bug`), a row (`code/train/6`, or `train/6` in the default config), a MiMo task's id (`format-code-task-000720`). An adapter may accept old refs and answer with the canonical one (`view["ref"]`): the page moves to it. Most formats don't need a new adapter: a dataset of rows needs a **row reader** (below). Write an adapter when tasks aren't rows, when they need an index of their own, or when the format has its own harness. ## An adapter ```python from app.envs import contract as c class MyAdapter(c.Adapter): id, name, framework = "mine", "My format", "Mine" # id: logs, URLs, run records; framework: the badge about = "One sentence: what a task is here, and how it's graded." def detect(self, kind, meta): # 0 to 1, cheap: the Hub's record (tags, card) return 0.9 if "my-format" in (meta.get("tags") or []) else 0.0 def confirm(self, env): # optional: a closer look when unsure (< 1) return True def summary(self, env, subset=None): return {"state": "ready", "total": 120, "inline": True, "search": True, "subsets": [], "how": {"framework": self.framework, "name": self.name, "about": self.about}, "facets": [c.facet("level", "Difficulty", "level")], "overview": [c.section("graded", "How it's graded", [c.kv([("Reward", "1 if the tests pass")])], icon="target")]} def tasks(self, env, *, subset=None, q="", filters=None, offset=0, everything=False): cards = [c.card(t.id, t.title, brief=t.brief, lead=t.group, facets={"level": [t.level]}) for t in load(env)] return {"cards": cards, "total": len(cards), "offset": 0, "page": len(cards)} def task(self, env, ref): t = find(env, ref) # LookupError -> 404 return {"ref": ref, "title": t.title, "chips": [t.level], "withheld": ["solution"], "sections": [c.section("task", "The task", [c.markdown(t.prompt)], icon="message"), c.section("files", "Files", [c.files(t.tree)], icon="folder")], "glance": [["Reward", "one score"]], "links": []} def run_options(self, env, ref, view): # [] and `run_note` when it can't run here return [c.run_option("harbor", "Harbor, on an HF Sandbox", ok=True, default=True)] def materialize(self, env, ref): # the task as a Harbor task folder, for the Harbor runner ... def aliases(self, env, ref): # [(environment key, ref)]: the same task elsewhere return [] ``` Put an instance in `ADAPTERS` (`app/envs/registry.py`); order doesn't matter, `detect` decides. While an adapter is still finding out (an index being built) it returns `False` from `settled` and its summary `{"state": "indexing", "progress": …}`; one that finds the source isn't its after all returns `{"state": "handoff"}` and the next adapter reads it. **Inline or paged.** `inline: True` means `tasks(everything=True)` returns every task, and the page searches and filters as you type (fuzzy, ranked, highlighted), with tiles and a map. Otherwise the page asks `tasks` for a page at a time, passing `subset`, `q`, `filters` ({facet: [values]}) and `offset`, and draws `facets` with the `values` you counted. **An inline list can say more:** `tiles` (`c.tiles(key, items)`: a big button per value of a facet, each with a note, an icon and a colour), facets with a `scope` (shown only with one of these tiles picked, `"*"` with none), a `map` (`c.treemap(by_tile={tile: facet})`: with no tile picked, every tile's top values side by side), `order: "shuffle"` (a fresh order each visit) and `noun` (`["environment", "environments"]`). An overview section can be `collapsed`. ## Blocks A section is a list of blocks, built with the helpers in `contract.py`. The page draws each kind (`web/js/blocks.js`) and MCP turns each into text (`app/envs/tools.py`). | block | for | drawn as | |---|---|---| | `kv(rows)` | label/value facts (values: inline Markdown or numbers; `shares(...)` too) | a definition list | | `shares(rows, of)` | parts of a whole | rows with bars | | `stats(rows)` | a few figures | a row of figure boxes | | `markdown(text)` | prose, the task as the agent gets it | rendered Markdown | | `code(text, path, label)` | a file shown inline | highlighted; "open in Files" when the files hold it | | `note(text, icon)` | a line of fine print | an icon and a sentence | | `value(v)` / `messages(v)` | any JSON / a chat | folding rows, images, audio, tables / a conversation | | `steps(items)` | a multi-step task | a numbered list | | `links(items)` | other pages (https) | buttons | | `disclose(label, blocks)` | something long | folded under a line | | `files(tree, hub=…)` | a task's files | the file viewer; serve contents from `file`, unlisted folders from `folder` | | `custom(renderer, kind, data, text)` | a view no block covers | `web/js/renderers/.js` | **Custom views.** Prefer blocks: they need no front-end code and agents read them as text. When a format needs a view of its own (MiMo's systems and their databases, its workspace previews, its grader), write a renderer module: `web/js/renderers/.js` exporting `render(kind, data, ctx)` (HTML) and optionally `wire(root, ctx)` (clicks). `ctx` has `spec` and `ref`; fetch anything more through your adapter's `data(env, ref, name, params)` (`/api/env///data?ref&part=&…`) and serve a task's files as they are with `raw(env, ref, path)` (`/raw?ref&f`, always with a sandboxing CSP). Give every custom block a `text`: it is what MCP and any page that can't load the renderer show. ## Running Every summary and task also receives a versioned `support` descriptor from `app/envs/capabilities.py`. It separates the source's framework, lifecycle, reward semantics, transport and available capabilities. The UI and dataset MCP tools consume the same descriptor. States are `available`, `external`, `unavailable`, and `unknown`; only an implemented hosted runner can advertise execution here. Register a new framework's documentation/lifecycle there and add its runner to `HOSTED_RUNNERS` only after implementing dispatch and validation. See [EXPLORER_AUDIT.md](EXPLORER_AUDIT.md) for the framework matrix and live protocol contract. A run option is one way to run a task, by a **runner**: - `harbor`: OpenEnv's Harbor runner on an HF Sandbox (`app/runner.py`). Implement `materialize` (the task as a Harbor folder) and `run_task` (title, sha, image, bytes, runnable). The agent is the visitor's pick (OpenCode, Terminus 2, mini-SWE-agent, Pi). - `mimo`: the MiMo release's own harness (`app/mimo/runner`): OpenCode in the task's image, Xiaomi's graders. `c.run_option(runner, label, ok=, why=, default=, about=, notes=, warnings=, fields=, harnesses=, endpoint=, sandbox=, estimate=)`. `fields` are the option's own inputs, each a `c.field(key, label, type, …)`: `text`, `number` (with `min`/`max`), `bool`, `select` (`options`), or `model` (a judge from `pool`, `"text_judges"` or `"vision_judges"`), `advanced=True` to fold it under Advanced settings, `required=True` when a run needs it. The server checks every field against the option the run picked. `endpoint` lets the visitor bring an OpenAI-compatible endpoint; `estimate` ({tokens_in, tokens_out, minutes, flavor}) prices a typical run on the panel. A new runner is a branch in `runner.submit` that writes a run record with `dataset`, `path` (the ref), `env`, `adapter` and `runner`, so its rollouts list with every other; if it records events instead of a trajectory, `run.js` hands its page over (see `run-mimo.js`). ## The same task in two places `aliases` says which other (environment, ref) pairs are this very task; their rollouts are listed with it, and the compare view takes rollouts from all of them. A relation neither adapter knows (a converted dataset and its source) goes in the registry instead: a **linker** `(env, ref, view) -> [c.link(label, href, note, rel="same")]` puts a link on the page, an **aliaser** `(env, ref) -> [(key, ref)]` merges the rollouts. `mimo.py`'s `harbor_to_mimo` and `harbor_aliases` link FineEnvs' Harbor conversions back to the MiMo release. ## Rules - **Never return an answer.** Leave out reference solutions, gold labels, a rubric's anchors, the code that checks them, anything that would let an agent read off the answer; name what you left out in `withheld`. Files that hold answers are listed with `withheld: True` and never served. Keys starting with `_` in a task view are yours: the registry strips them before anything leaves (use them to hand `run_options` what it needs). - **Use the visitor's token only for private sources** (`env.read_token`); public data is read anonymously and cached for everyone. - **Be quick, or say you're working.** A slow first read belongs behind an index (with `progress`) or a background load; pages poll. ## Row readers A dataset of rows (Verifiers, NeMo Gym, verl, agent traces, Harbor tasks packed into rows, any table of prompts) needs no adapter: `RowsAdapter` reads it through the Hub's dataset viewer (pages, filters, statistics for facets, search), or, when the viewer can't, straight from its JSON Lines / JSON / parquet / CSV files. A **processor** (`app/envs/processors.py`) says how one row becomes a task; the generic one infers each column's role from its name and values. Write one when a format deserves more: a nested request to unpack, an archive to open, a grader to explain, a twin to run. ```python from app.envs.base import Dataset, Processor, section, withhold, parse_json, text_of, first_line class MyRows(Processor): id, name, framework = "my-rows", "My rows", "My rows" about = "One sentence: what a row is and how it's graded." def match(self, ds: Dataset) -> float: # the generic reader scores 0.1 return 0.9 if ds.has("my_request", "my_checks") else 0 def view(self, row, i, ds, roles): clean, gone = withhold({k: parse_json(v) for k, v in row.items()}) # answers out, by name, at any depth return {"title": first_line(text_of(row["my_request"])) or f"Row {i}", "id": str(row.get("id")), "sections": [section("task", "The task", row["my_request"]), section("grading", "How it's graded", clean["my_checks"], kind="value")], "withheld": gone, "glance": [["Row", f"{i:,}"]], "run": None} # or {"dataset": "org/harbor-twin", "path": "tasks/x"}: a twin that runs here ``` Add it to `PROCESSORS`. Section kinds: `markdown`, `messages`, `value`, `code` ({path, text}), `files` ({tree}, serve each with `file(row, ds, rel)`) and `blocks` (a list of contract blocks, for a reader that builds its own: `custom` views, `kv`, `links`, ...); the rows adapter turns them into blocks. A row's ref is `/`, or `//` outside the default config. A processor can say more than a row: - `overview(ds, roles, stats, config, split)`: sections for the dataset's own page (what a row asks, how it's graded, how to run it), from the first rows (`ds.sample`) and the dataset viewer's column statistics (`stats`, `[]` when the rows come from files). Don't read more than that: it runs on every dataset page. - `framework_for(ds)`: the badge, when one reader covers two formats (verl and SkyRL). - In `view`: `framework_run`, a `c.run_option(...)` with `ok=False` that says in one sentence why the task can't run here and points at the page's run section (the run panel shows it; the runner refuses it); `summary`, a line under the title (what scores it); `framework`, this row's badge. - `withhold(value, hide=frozenset({...}))`: answers whose names don't say so ("expected_action", "verifier.patterns"), as dotted names, through lists. Put a format's set in `processors.HIDDEN` too: search never looks at them. **The framework readers.** NeMo Gym, Verifiers and verl / SkyRL rows are read by modules of their own, which show a row as its framework sees it and say how to run it there. Verifiers, verl / SkyRL and most NeMo configurations require their native servers, sandboxes or trainers. The two supported NeMo Pivot tool-simulation configurations can also run here through Responses inference and NVIDIA's pinned argument verifier; they predict function calls without executing the functions. `nemogym.run_option` validates this narrow hosted subset: | module | knows | how it's kept current | |---|---|---| | `nemogym.py` | every agent and resources server NeMo Gym's configs define, what each verifies, which row fields its verifier reads and which hold the answer (`nemo_gym_catalog.json`) | rebuild from a checkout: `uv run python -m app.envs.nemo_gym_build /path/to/Gym` (reviewed overrides in `EXTRA_HIDE` / `NOT_SECRET`) | | `verl_rows.py` | verl's `default_compute_score` routing (`data_source` → scorer), SkyRL's `env_class` registry, the trainers' commands | `SCORERS`, `SKYRL_ENVS`, `run_blocks` | | `verifiers_rows.py` | Prime's datasets and the Hub environment that loads each, their API (v0/v1, asked of the Environments Hub), SWE grading material to withhold | `KNOWN`, `DEFAULTS`, `SWE_HIDE`, `DATASET_HIDE` | They draw with blocks and one renderer, `web/js/renderers/rl.js`: `transcript` (a conversation as the policy gets it: messages, folded reasoning, tool calls with their outputs paired by call id; a long middle folds), `tools` (each tool as a signature: parameters, types, required, allowed values) and `fields` (a row's other fields, label over value). `convo.py` reads Responses API items and chat messages into `turns`, tool definitions into signatures, and makes titles (`task_title`: an issue's own title, past a template's preamble). | processor | reads | example | |---|---|---| | `harbor-packed` | a Harbor task packed into each row (tar/tar.gz/zip, base64 or bytes); runs | open-thoughts/TaskTrove | | `mimo` | MiMo-V2.6 RL rows (verl-shaped, with an instance packed in `extra_info`) | copies of the release | | `nemo-gym` | `responses_create_params` (the conversation, the tools), `agent_ref` (→ its resources server), the verifier's task data; Pivot rows' expert step | nvidia/Nemotron-RL-* | | `verl` | prompt, data_source (→ verl's scorer), ability, reward_model, extra_info; SkyRL's env_class and reward_spec | sungyub/*-verl, NovaSky-AI/SkyRL-SQL-* | | `traces` | recorded rollouts: the agent's turns, its result | open-thoughts/AgentTrove | | `verifiers` | data for a Verifiers environment: v0 question/prompt/answer/info, Prime's SWE tasksets | PrimeIntellect/* | | `generic` | anything else | FineEnvs/SmolDataEnvs | ## Trying one ```bash uv run python -c " from app.envs import registry s = registry.summary('org/my-dataset'); print(s['env'], s['total'], [f['key'] for f in s['facets']]) r = registry.tasks('org/my-dataset', everything=True)['cards'][0]['ref'] print(registry.task('org/my-dataset', r)['sections'])" ``` Then open `http://localhost:8060/d/org/my-dataset`, connect an agent to `/mcp/d/org/my-dataset`, and add tests: `tests/test_contract.py` registers a toy adapter and checks the whole contract end to end (pages, runs, aliases, MCP); `tests/test_mimo.py` and `tests/test_envs.py` show an adapter's and a processor's own checks, answers first.