FrontierLab-Meta-Hackathon-2026 / docs /RESEARCH_OPERATING_INTERFACE.md
Jaycee766's picture
Deploy FrontierLab v0.38 source 64c7aa4458d6
0cca430 verified
|
Raw History Blame Contribute Delete
5.64 kB

Research Operating Interface β€” conceptual architecture

Purpose

The Research Operating Interface is FrontierLab's Information Management view of scientific agents.

It treats papers, code, data and agent tools as managed research resources and adds three control functions:

                    Research Operating Interface
                              |
          ---------------------------------------------
          |                    |                      |
   Research artefacts      Agent interfaces      IM control layer
   Paper / Code / Data   MCP Tools/Resources/     Maintainability
                              Prompts              Provenance
                                                   Governance

This document is a concept architecture, not a claim that every module below is already deployed.

Module map

Module Information-management role FrontierLab status
Paper literature/knowledge input implemented for current biomedical literature workflow
Code executable research method represented in reproducibility evaluation; general code orchestration is future work
Data datasets / APIs literature/source data implemented; general institutional data integration is future work
MCP Tools executable agent capabilities conceptual/integration direction
MCP Resources managed references/data resources conceptual/integration direction
MCP Prompts reusable task contracts analogous structured role/task contracts exist; generalized MCP prompt management is future work
Evidence Trace claim-to-source provenance implemented through Evidence Graph + measured Trace Coverage
Process Trace who/what happened during execution implemented through Trajectory Audit Graph + human events
Governance permissions, approvals, accountability partial prototype through human gates, budgets and audit rules
Maintainability failure detection, regression, rebuild/repair scheduled maintenance canary + dependency fingerprint + quarantine policy + advisory repair proposals implemented; autonomous repair/rebuild remains disabled

Relationship to the current five-stage pipeline

The interface is a management layer over the current workflow:

Research Operating Interface
        |
        +-- Research materials / tools / policy
        |
        +--> FrontierLab scientific workflow
             Research
               ↓
             Evidence
               ↓
             Hypothesis
               ↓
             Evaluation
               ↓
             Planning

Observable interface concepts

Current Activity

Expose the current stage, active sources, tool/backend identity, blocked state and human-review status.

Active Sources

Show the sources currently contributing to the run.

A future multi-source interface may include PubMed/Europe PMC, arXiv, IEEE, Nature or institutional systems, but FrontierLab must display only sources actually connected in a given deployment.

Evidence Graph

Display relationships between sources, passages, claims, hypotheses and plan steps.

Source Audit

Expose citation/source existence and quotation anchoring checks.

Trace Coverage

Implemented in frontierlab/traceability.py and exposed through the Research Operating Interface.

Two values are reported:

  • Source Trace Coverage β€” fraction of in-scope downstream artifacts that reach at least one known Paper;
  • Strict Extracted Coverage β€” fraction whose complete reachable path is EXTRACTED, resolves to Paper nodes, and has no unresolved sources or dead ends.

Recorded reviewed M2 core measurement:

  • 4 / 4 in-scope artifacts source-traceable = 100% Source Trace Coverage;
  • 2 / 4 fully EXTRACTED = 50% Strict Extracted Coverage.

The difference is intentional. The PMC4314675 path preserves an original model stance of inconclusive even though the later researcher review judged the claim SUPPORTED, so that path remains AMBIGUOUS rather than being rewritten into strict evidence.

These are provenance metrics, not model accuracy or scientific truth. See Trace Coverage.

Governance panel

Expose:

  • permissions;
  • approval requirements;
  • model/tool budgets;
  • sensitive-data policy;
  • reviewer responsibility;
  • prohibited actions.

Maintainability panel

Phase 1 is now operationalized by the scheduled Maintenance Canary.

The canary records:

  • Python and tracked package versions;
  • requirements-file hashes;
  • a dependency fingerprint;
  • full regression-test outcome;
  • HEALTHY / QUARANTINE_REQUIRED status;
  • whether promotion is permitted.

The canary also emits an advisory repair proposal by comparing the current health record with a human-controlled accepted baseline. It can distinguish stable health, healthy drift and several failure classes, but it cannot edit code/dependencies, commit, merge or deploy.

Automatic repair and environment rebuild remain disabled. A failing canary preserves its evidence and requires human-approved repair before promotion.

See Maintenance canary and Advisory maintenance repair proposals.

Design principle

The interface should answer three questions at any moment:

  1. What is the system doing?
  2. Where did its scientific output come from?
  3. Who or what authorized the next action?

For a maintainable system, it should eventually answer a fourth:

  1. If a tool breaks tomorrow, can we detect, reproduce and repair the failure without losing scientific provenance?