app/src/content/chapters/02-extraction.mdx

import HtmlEmbed from '../../components/HtmlEmbed.astro';
import Wide from '../../components/Wide.astro';
import Note from '../../components/Note.astro';
import Sidenote from '../../components/Sidenote.astro';

# From Traces to Symbols

An agent execution trace is a sequence of messages $\tau = (m_1, m_2, \ldots, m_T)$, where each message has a **role** (system, user, assistant, tool) and **content**. The first step is to map each message to a symbolic activity from a finite alphabet.

## Activity Extraction

An *activity extraction function* $\phi: m_t \mapsto a_t \in \mathcal{A}$ maps each message to a symbol. We apply three rules in priority order:

1. **Tool calls**: if a message contains a `tool_call` field, the activity is the function name (for example `bash`, `search_flight`, `click`).
2. **Action tags**: if the content contains `[ACTION] description`, the activity is the action label.
3. **Command extraction**: for agents that act through code blocks, we extract the first command token and map it to a semantic category (`edit`, `search`, `navigate`, `execute`).

If no rule matches, the activity defaults to `role:content_type` (for example `assistant:text`).

<Note>
The extraction is entirely deterministic and format-specific, with no LLM calls. The whole process completes in milliseconds.
</Note>

## An Example

Consider a coding agent trace from SWE-agent with 47 messages. The raw trace contains system prompts, file contents, error messages, and tool invocations. After extraction, the activity sequence is:

<p style="text-align:center; line-height:2.2;">
<code>init</code> → <code>user</code> → <code>search</code> → <code>user</code> → <code>edit</code> → <code>user</code> → <code>execute</code> → <code>user</code> → <code>edit</code> → <code>user</code> → <code>submit</code>
</p>

From 47 messages and thousands of tokens, we get 11 symbols drawn from an alphabet of 24 possible activities. This is the sequence the FSM will model.

<Wide>
<HtmlEmbed
  src="embeds/trace-explorer.html"
  title="Trace Explorer"
  caption="Step through an agent trace to see how raw messages map to symbolic activities. Each message is classified by the extraction rules into one of the agent's activity symbols."
/>
</Wide>

Replay any real trace symbol by symbol in the [trace view](https://seongland.com/article/asg/browser?tab=traces&dataset=sweagent).

## Why This Works

These alphabets are small by construction: 6 to 42 symbols across the twelve datasets, against the tens of thousands of natural language. That is what keeps FSM extraction tractable.

Even a 42-tool telecom customer service agent (tau2-bench telecom) needs only 43 states to capture its behavioral structure. The alphabet is bounded because the agent's capabilities are bounded: it can only call the tools it has been given.

<Note variant="info" title="How sensitive is the result to this choice?">
$\phi$ is the one design decision in the pipeline, so we stress-test it. Fitness stays above 0.999 across all four granularities, from role-only (two to four symbols) to full tool-level. Failure prediction stays within 0.03 [AUROC](https://texonom.com/c3fea8b9caa445768eb529fb629818c7) (area under the ROC curve) on any dataset. The rules above are one valid setting, not the only one.
</Note>