Add an Inspect @tool example and the carbon-dna skill for coding agents

README.md

@@ -324,9 +324,97 @@ while True:
 print(message.content)
 ```
 
-Coding agents such as Claude Code or Codex CLI can use the same function as a skill: a short `SKILL.md` that says when to consult Carbon-X, plus a script that calls `ask_carbon` on sequence files in the workspace.
 
-Tips for agent use:
+### With Inspect
+
+In [Inspect](https://inspect.aisi.org.uk/), wrap Carbon-X in an `@tool` and give it to a `react()` agent. Each sample
+keeps its DNA in metadata, so the agent refers to sequences by name.
+
+```python
+from inspect_ai import Task, eval, task
+from inspect_ai.agent import as_solver, react
+from inspect_ai.dataset import Sample
+from inspect_ai.scorer import includes
+from inspect_ai.solver import solver
+from inspect_ai.tool import ToolError, tool
+from inspect_ai.util import store
+from openai import AsyncOpenAI
+
+carbon = AsyncOpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
+
+
+@tool
+def ask_carbon():
+    async def execute(question: str, sequences: list[str], context: str = "") -> str:
+        """Ask Carbon-X, a DNA language model, about this task's DNA sequences. Its answers are predictions.
+
+        Args:
+            question: One self-contained question.
+            sequences: Names of the sequences to send, as given in the task.
+            context: Known facts, such as the gene, positions or alleles.
+        """
+        dna = store().get("sequences")
+        if missing := [name for name in sequences if name not in dna]:
+            raise ToolError(f"Unknown sequence names: {missing}. Available: {list(dna)}")
+        parts = [context, question] if context else [question]
+        parts += [f"{name}: <dna>{dna[name]}</dna>" for name in sequences]
+        response = await carbon.chat.completions.create(
+            model="HuggingFaceBio/Carbon-X",
+            messages=[{"role": "user", "content": "\n\n".join(parts)}],
+            max_tokens=4096,
+            extra_body={"chat_template_kwargs": {"enable_thinking": False}},
+        )
+        return response.choices[0].message.content
+
+    return execute
+
+
+@solver
+def load_sequences():
+    async def solve(state, generate):
+        store().set("sequences", state.metadata["sequences"])  # the tool reads DNA from here
+        return state
+
+    return solve
+
+
+@task
+def carbon_agent():
+    sample = Sample(
+        input="Sequence `variant` differs from `reference` at position 20 (A>T). Is the variant benign or pathogenic?",
+        target="pathogenic",
+        metadata={"sequences": {
+            "reference": "ATGGTGCATCTGACTCCTGAGGAGAAGTCTGCCGTTACTGCCCTGTGGGGCAAGG",
+            "variant": "ATGGTGCATCTGACTCCTGTGGAGAAGTCTGCCGTTACTGCCCTGTGGGGCAAGG",
+        }},
+    )
+    agent = react(prompt="Consult Carbon-X with the ask_carbon tool before you answer.", tools=[ask_carbon()])
+    return Task(dataset=[sample], solver=[load_sequences(), as_solver(agent)], scorer=includes())
+
+
+eval(carbon_agent(), model="openai/gpt-5")  # any Inspect model with tool calling as the agent
+```
+
+### As a skill for coding agents
+
+This repository includes a ready-made skill in [`skills/carbon-dna/`](skills/carbon-dna/): a `SKILL.md` that tells the
+agent when and how to consult Carbon-X, and a standard-library Python script with two commands, `ask` (a question about
+named DNA files) and `compare` (rank equal-length variants by sequence likelihood).
+
+```bash
+hf download HuggingFaceBio/Carbon-X --include "skills/*" --local-dir carbon-x
+cp -r carbon-x/skills/carbon-dna ~/.claude/skills/    # Claude Code; Codex CLI: ~/.codex/skills/
+export CARBON_BASE_URL=http://localhost:8000/v1        # your Carbon-X server
+```
+
+The script also runs on its own:
+
+```bash
+python carbon-x/skills/carbon-dna/scripts/carbon.py ask --sequence reference=ref.fa --sequence variant=alt.fa \
+  --question "Is the variant benign or pathogenic?" --context "Position 20: A>T"
+```
+
+### Tips for agent use
 
 - **Pass DNA by reference.** Keep sequences in files or a lookup table and let the agent name them.
 - **Ask for Carbon-X explicitly.** Agents often answer DNA questions on their own instead of calling a tool. If you want Carbon-X's view, say so in the system prompt.

skills/carbon-dna/SKILL.md

@@ -0,0 +1,42 @@
+---
+name: carbon-dna
+description: "Consult Carbon-X, a DNA language model, about DNA sequences in files: variant effects, regulatory elements, organism of origin, sequence features, and likelihood comparisons between variants."
+---
+
+# Consult Carbon-X about DNA
+
+Carbon-X is a DNA language model. Use it for biological interpretation of DNA alongside your own analysis. Its answers
+are model predictions, not verified facts or clinical classifications.
+
+`scripts/carbon.py` (next to this file) talks to a server running Carbon-X. It needs only Python 3.10+. The user
+configures the server with `CARBON_BASE_URL` (default `http://localhost:8000/v1`), `CARBON_MODEL` and, if needed,
+`CARBON_API_KEY`. Resolve the script's absolute path from this skill's directory.
+
+Put each sequence in its own file (plain DNA or one FASTA record) and refer to files by name. Do not retype DNA into
+commands: copying long sequences by hand introduces errors.
+
+Ask a question about one or more sequences:
+
+```sh
+python3 /path/to/carbon-dna/scripts/carbon.py ask \
+  --sequence reference=reference.fa --sequence variant=variant.fa \
+  --question "variant differs from reference at one base. Is the change likely benign or pathogenic?" \
+  --context "Position 20 (1-based): A>T. Gene and coordinates, if known."
+```
+
+Rank equal-length variants by how typical Carbon-X finds each sequence (a likelihood, not a pathogenicity score):
+
+```sh
+python3 /path/to/carbon-dna/scripts/carbon.py compare --reference reference.fa --alternative variant=variant.fa
+```
+
+From Python, add `scripts/` to `sys.path` and call `ask(question, {name: dna}, context)` or
+`compare(reference_dna, {name: dna})`.
+
+Guidance:
+
+- Work out exact facts in code first (positions, reference and alternate bases, reading frame, lengths) and pass them
+  as `--context`. Carbon-X can miscount positions in long sequences.
+- Ask one self-contained question per call, and keep any answer choices from the user's question.
+- Carbon-X reads A/C/G/T only. The script trims `N` padding at the ends and refuses other characters.
+- Check `truncated` in the result; for next-base or continuation questions, ask for only the length you need.

skills/carbon-dna/scripts/carbon.py

@@ -0,0 +1,105 @@
+"""Ask Carbon-X about DNA files, or compare DNA variants by model likelihood.
+
+Talks to any OpenAI-compatible server running Carbon-X, such as `vllm serve HuggingFaceBio/Carbon-X --trust-remote-code`.
+Configure it with environment variables:
+    CARBON_BASE_URL  server URL including /v1 (default http://localhost:8000/v1)
+    CARBON_MODEL     served model name (default HuggingFaceBio/Carbon-X)
+    CARBON_API_KEY   bearer token, if the server needs one (for example an HF token for an Inference Endpoint)
+
+Usage:
+    python carbon.py ask --question "..." --sequence reference=ref.fa --sequence variant=alt.fa [--context "..."]
+    python carbon.py compare --reference ref.fa --alternative alt1=alt1.fa [--alternative alt2=alt2.fa]
+Only the Python standard library is needed. Output is JSON.
+"""
+
+import argparse
+import json
+import os
+import re
+import sys
+import urllib.request
+from pathlib import Path
+
+BASE_URL = os.environ.get("CARBON_BASE_URL", "http://localhost:8000/v1").rstrip("/")
+MODEL = os.environ.get("CARBON_MODEL", "HuggingFaceBio/Carbon-X")
+API_KEY = os.environ.get("CARBON_API_KEY", "")
+
+
+def read_dna(path: str) -> str:
+    """Read plain DNA or one FASTA record; keep only A/C/G/T after trimming N padding at the ends."""
+    lines = [line.strip() for line in Path(path).read_text().splitlines() if line.strip()]
+    if lines and lines[0].startswith(">"):
+        lines = lines[1:]
+    if any(line.startswith(">") for line in lines):
+        sys.exit(f"{path}: one sequence per file, please (multi-record FASTA found).")
+    dna = "".join(lines).upper().strip("N")
+    if not re.fullmatch("[ACGT]+", dna):
+        sys.exit(f"{path}: Carbon-X reads A/C/G/T; remove other characters first.")
+    return dna
+
+
+def post(route: str, payload: dict) -> dict:
+    request = urllib.request.Request(f"{BASE_URL}/{route}", json.dumps({"model": MODEL, **payload}).encode(),
+                                     {"Content-Type": "application/json", **({"Authorization": f"Bearer {API_KEY}"} if API_KEY else {})})
+    with urllib.request.urlopen(request, timeout=600) as response:
+        return json.load(response)
+
+
+def ask(question: str, sequences: dict[str, str], context: str = "", max_tokens: int = 4096) -> dict:
+    """Ask a question about named DNA sequences. Returns Carbon-X's answer, a model prediction."""
+    parts = [context, question] if context else [question]
+    parts += [f"{name}: <dna>{dna}</dna>" for name, dna in sequences.items()]
+    response = post("chat/completions", {"messages": [{"role": "user", "content": "\n\n".join(parts)}],
+                                         "max_tokens": max_tokens, "chat_template_kwargs": {"enable_thinking": False}})
+    choice = response["choices"][0]
+    return {"answer": choice["message"]["content"], "truncated": choice["finish_reason"] == "length"}
+
+
+def log_likelihood(dna: str) -> float:
+    """Total log-likelihood (nats) of the DNA tokens of `<dna>{dna}</dna>` under Carbon-X."""
+    response = post("completions", {"prompt": f"<dna>{dna}</dna>", "max_tokens": 1, "temperature": 1.0,
+                                    "prompt_logprobs": 0, "add_special_tokens": False, "return_token_ids": True})
+    choice = response["choices"][0]
+    ids, logprobs = choice["prompt_token_ids"], choice["prompt_logprobs"]
+    n = (len(dna) + 5) // 6  # one token per 6 bases; a shorter tail gets its own token
+    if len(ids) != n + 2:  # <dna>, n DNA tokens, </dna>
+        sys.exit("The server did not tokenize the DNA as expected; is it serving Carbon-X with --trust-remote-code?")
+    return sum(logprobs[i][str(ids[i])]["logprob"] for i in range(1, n + 1))
+
+
+def compare(reference: str, alternatives: dict[str, str]) -> dict:
+    """Rank equal-length alternatives by log-likelihood relative to the reference (higher is more typical DNA).
+
+    This is sequence likelihood, not a calibrated pathogenicity score.
+    """
+    if any(len(dna) != len(reference) for dna in alternatives.values()):
+        sys.exit("compare needs equal-length sequences; for insertions or deletions, use ask.")
+    base = log_likelihood(reference)
+    rows = [{"name": name, "delta_log_likelihood": log_likelihood(dna) - base,
+             "substitutions": sum(a != b for a, b in zip(reference, dna))} for name, dna in alternatives.items()]
+    return {"reference_log_likelihood": base, "alternatives": sorted(rows, key=lambda r: -r["delta_log_likelihood"])}
+
+
+def named(items: list[str]) -> dict[str, str]:
+    pairs = [item.split("=", 1) for item in items]
+    if any(len(pair) != 2 for pair in pairs):
+        sys.exit("Use NAME=FILE for each sequence.")
+    return {name: read_dna(path) for name, path in pairs}
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    sub = parser.add_subparsers(dest="command", required=True)
+    a = sub.add_parser("ask")
+    a.add_argument("--question", required=True)
+    a.add_argument("--sequence", action="append", default=[], metavar="NAME=FILE")
+    a.add_argument("--context", default="")
+    c = sub.add_parser("compare")
+    c.add_argument("--reference", required=True, metavar="FILE")
+    c.add_argument("--alternative", action="append", required=True, metavar="NAME=FILE")
+    args = parser.parse_args()
+    if args.command == "ask":
+        result = ask(args.question, named(args.sequence), args.context)
+    else:
+        result = compare(read_dna(args.reference), named(args.alternative))
+    print(json.dumps(result, indent=2))