onf-c1/docs/technical/05-artifacts.md

05. Artifacts

Reads: every .npz in this repo Writes: nothing — reference chapter Code: src/onf/graph/core/schema.py (filenames), src/onf/graph/core/geometry.py::graph_hash, src/onf/graph/run/params.py, src/onf/config.py::Paths Stage: BUILD and RUNTIME Read after: 03-action-chunk-blend.md

Every artifact the repo produces or consumes, who writes it, who reads it, and the one check that stops the worst failure in the subsystem.

---

5.1 The table

BUILD = an offline build/train script touches it. RUNTIME = an eval or deploy process reads it while an episode is running. The shipped runtime regime is sentinel (onf.sentinel).

artifact	produced by	read by	build / runtime	regime
q_flow.npz	onf.field.build (build_qflow, build_suite)	onf.field.train.DemoCloud.load	BUILD only	indirectly via onf_head.npz
q_home.npz	onf.field.build (_save_home, from both entry points)	nothing	dead — written, never loaded	neither
onf_head.npz	onf.field.train.train_onf (from q_flow.npz)	onf.field.field.ONFField.load, reached via onf.graph.core.geometry.load_cleanliness_field	RUNTIME	field_dir is threaded through sentinel/sentinel.py into GraphRetriever.load
q_manifold.npz	onf.field.build.SuiteBuilder.build_suite (pre-grasp slice)	nothing	dead — written, never loaded	neither
q_manifold_full.npz	onf.field.build.SuiteBuilder.build_suite (full trajectories)	nothing	dead — written, never loaded	neither
g_nodes.npz	onf.graph.build.from_demos (NodeTable.save, name from schema.NODES_NPZ)	GraphRetriever.load (NodeTable.load), onf.graph.train.loop, scripts/build_sentinel_artifacts.py	RUNTIME	every retriever/tracker load reads it
g_edges.npz	onf.graph.build.from_demos (EdgeSet.save, schema.EDGES_NPZ)	same as g_nodes.npz	RUNTIME	as above
g_head.npz	onf.graph.train.loop.save_checkpoint (schema.HEAD_NPZ), graph_hash-stamped	GraphRetriever.load, scripts/build_sentinel_artifacts.py, scripts/capture_golden.py; onf.graph.cli checks only that it exists	RUNTIME	the trained GNN retrieval head
g_track.npz	scripts/build_sentinel_artifacts.py via onf.graph.run.params.TrackParams.save (schema.TRACK_NPZ), graph_hash-stamped	onf.graph.run.track.GraphTracker.load → TrackParams.load	RUNTIME	sentinel
sentinel_artifacts.json	scripts/build_sentinel_artifacts.py	nothing in code	BUILD-time sidecar	sentinel (documents how g_track.npz was fit)
action_scale.json	scripts/fit_action_scale.py	onf.blend.ee_track.ActionScale.from_json, reached from Sentinel._action_scale	RUNTIME	blend — metres/radians per action unit. Mandatory for any blend mode; absence raises
a_pi_raw.npz	scripts/cache_policy_actions.py	onf.blend.target.PolicyActionCache	BUILD only	the frozen policy's own chunks, needed to train the blend weight
g_alpha.npz	onf.graph.train.loop under --objective chunk, via onf.blend.alpha.AlphaNet.save	AlphaNet.load, reached from Sentinel._alpha_net	RUNTIME	blend — only under SN_BLEND_LEARNED; absence raises rather than falling back to a fixed weight

g_track.npz holds the fitted belief-filter transition kernel (pi / beta / leak) and the basin geometry (basin_r / basin_h), all f8. It is fit by onf.graph.build.tracker_fit (build_likelihood_sequences, fit_transition_kernel) from the graph alone — no simulator, no success-rate outcome. Files written before the e-process null banks were retired still carry those keys; the loader ignores them.

g_track.npz is the one optional artifact. GraphTracker.load checks whether the file exists; if it does not, the tracker still loads on documented unfit defaults (uniform advance kernel) rather than crashing. Every other artifact in the table is mandatory for its regime. A g_track.npz that is present but stale is refused — §5.2.

Where they live

group	directory	resolver	override
field: q_flow, q_home, onf_head, q_manifold*	data/fwm/<suite>/	Paths.fwm(suite)	QNDF_DIR
graph: g_nodes, g_edges, g_head, g_track, sentinel_artifacts.json	outputs/<suite>/latest/artifacts/	Paths.graph(suite)	GR_GRAPH_DIR
blend: action_scale.json, a_pi_raw.npz, g_alpha.npz	beside the graph artifacts	Paths.graph(suite)	GR_GRAPH_DIR; SN_ACTION_SCALE for the calibration alone

The object suite's field lives at the fwm root, not at fwm/object — the mapping is read from configs/suites.yaml, not assumed. latest is a symlink repointed by onf.graph.report.StageLogger at every new build.

---

5.2 graph_hash — the most important check in the subsystem

src/onf/graph/core/geometry.py::graph_hash(nodes, edges) is a SHA-256 over V, E, dim and then, in order:

hashed	fields
node	q, qdot, owner, task_id, stage, t_idx, phase
edge	src, dst, rel, indptr

w and log_idf are excluded: they are pure functions of the hashed columns, so hashing them adds nothing.

Why this check, and not any other

Node ids are positional. A node id means "row 14,203 of this g_nodes.npz" and nothing more. Two builder runs over the same demos with one parameter changed — a different COARSEN, one extra demo, a re-sorted HDF5 directory — renumber every node.

Now load a head trained on graph A against graph B:

• the head's parameter tensors have no dimension equal to |V| (the inductivity guarantee, chapter 01 §1.6), so there is no shape mismatch;
• the forward pass runs to completion over B's nodes and returns a well-formed [V] logit vector;
• the softmax is peaked, the abstain logit is calibrated-looking, the readout returns a joint config inside the workspace;
• no NaN, no exception, no warning.

The output is confident, correctly shaped, and points at the wrong demonstration strands. Its end-effector segment then becomes a tracking chunk that is mixed straight into the actions a real arm executes (chapter 03) — and row 0 of that chunk is a pose error, so the further wrong the target is, the harder the blend drives towards it. Nothing downstream is capable of noticing: every check in the pipeline is a shape check, a finiteness check or a reachability mask, and a foreign head passes all three.

The same argument applies with more force to g_track.npz. The advance operators S_a are built by raw-frame arithmetic over node ids (NodeTable.node_at_raw), so a foreign kernel advances the belief along the wrong strands and projects onto basin geometry fitted on a different graph. The target and the basin no-op decision are then both wrong, consistently and silently.

So both files carry a graph_hash key, and both loaders refuse on mismatch:

loader	file	on mismatch
GraphRetriever.load	g_head.npz	ValueError, before any network weight is touched
TrackParams.load	g_track.npz	ValueError, before the kernel is built

A file predating the stamp reports "<none: head predates graph_hash stamping>" and is refused on the same path — absence is treated as mismatch, not as permission.

schema.NpzSchema.hash_stamped is the declaration: HEAD and TRACK are True, NODES and EDGES are False (they are the hash input, so there is nothing to check them against).

The three blend artifacts carry no stamp. action_scale.json is a property of the robot and the controller rather than of the graph, and g_alpha.npz / a_pi_raw.npz predate any decision about stamping them. AlphaNet.load therefore accepts a head trained against a different graph, and its failure mode is the mild version of the one above — the weight is wrong, not the target. The calibration is the one worth watching: a wrong action_scale.json scales every blended action, and the only guard is that a missing file raises rather than defaulting.

---

5.3 sentinel_artifacts.json — what it certifies, and what it does not

Written next to g_track.npz by scripts/build_sentinel_artifacts.py. Nothing in the code reads it back; it is a provenance record so a later run can be checked against exactly what an earlier g_track.npz was built from.

It records:

key	content
graph_hash, head_npz, graph_dir	which graph and which head the fit ran against
n_nodes, n_edges, n_demos, n_tasks, device	graph shape
split	the FIT / HELD-OUT owner partition rule, held_out_stride, fit_clean_stride, both set sizes, disjoint_fit_vs_held_out
fit_sequences	counts per kind (clean / drift / cross_strand) and seed
fit_transition_kernel	pi, beta, leak, beta_is_zero, log_evidence, the uniform-pi baseline and the delta, n_checks, n_sweeps, elapsed
basin summary	basin_r / basin_h ranges, backfilled-cell count, spacing

The split is deterministic. owner % HELD_OUT_STRIDE == 0 is held out; everything else is a FIT owner. A stride partition, no RNG, exactly reproducible. With owners assigned in contiguous per-task blocks — as this repo's builder does — the stride spreads held-out demos evenly across tasks instead of clumping them into one.

What it does not certify. split.head_training_disjointness is written verbatim as "not verified (no recorded head train/held-out split in this repo's graph-build pipeline)". The FIT / HELD-OUT disjointness is a guarantee about the kernel fit relative to the probe set. It says nothing about either relative to the demos that trained g_head.npz, because the graph-build pipeline records no such split for the GNN. The field is spelled out in the JSON specifically so nobody reads a stronger claim into it.

beta_is_zero is written unconditionally and warned about on the console. A fitted beta == 0.0 would contradict schema.py's "sibling mixing is mandatory"; the script reports it verbatim rather than papering over it.

---

5.4 q_manifold.npz / q_manifold_full.npz are dead

Both are written by SuiteBuilder.build_suite (src/onf/field/build.py, the manifold_full.save / manifold_pre.save pair). Neither has a reader.

Re-verified against today's tree. Every np.load / np.savez* call site in src/, tests/, scripts/ and evals/ was enumerated, and every textual occurrence of manifold was checked. The only surviving references to either filename are docstrings:

file	mention
src/onf/graph/core/nodes.py (×2)	contrasts the q_manifold_full.npz phase convention (/T) against NodeTable's (/(T−1)) — the phase_conv stamp check
src/onf/graph/train/data.py	cites a procedure the deleted build_qmanifold_full used
src/onf/field/build.py	the builder's own docstrings

None of them loads a file. The standalone build_qmanifold_full CLI function no longer exists; build_suite still writes both files inline. They are not deleted from disk here because data/ is a shared symlink (§5.6), and build_suite is not changed to stop writing them because that is a behaviour change outside this documentation pass.

---

5.5 How the pieces connect

raw demos ──field.build──▶ q_flow.npz ──field.train──▶ onf_head.npz ──ONFField.f_value──┐
                        └▶ q_home.npz ─────────────────────────────────────────┐        │
                                                                                │        │
raw demos ──graph.build.from_demos──▶ g_nodes.npz, g_edges.npz                 │        │
                                            │                                   │        │
                                            └──graph.train.loop──▶ g_head.npz   │        │
                                                                   │            │        │
                                                                   ▼            │        ▼
                                              onf.graph.run.retrieve.GraphRetriever
                                              (cleanliness() pools the query window,
                                               weighted by f_value)
                                                                   │            │
                                                                                ▼
                                                                   SENTINEL (onf.sentinel)
                                                        Sentinel._build_tracker passes field_dir to
                                                        GraphTracker.load, which additionally reads
                                                        g_track.npz — built offline by
                                                        scripts/build_sentinel_artifacts.py from
                                                        g_nodes/g_edges/g_head, no simulator, no SR
                                                                   │
                                                                   ▼
                                                        BLEND (onf.blend), reading
                                                        action_scale.json and, under
                                                        SN_BLEND_LEARNED, g_alpha.npz —
                                                        trained from a_pi_raw.npz + the graph

q_manifold.npz and q_manifold_full.npz are off this chain entirely.

---

5.6 outputs/ and data/ are shared symlinks

At this checkout's root:

data     -> ../vla-bottleneck/data
outputs  -> /srv/data/VR-SmallVLA/onf/outputs
results  -> ../vla-bottleneck/results

All three point into trees shared with sibling worktrees and checkouts. A rebuild started from another branch can overwrite q_flow.npz, onf_head.npz, g_nodes.npz, g_edges.npz, g_head.npz or g_track.npz underneath an eval that is reading them from this checkout.

The hash stamp covers only half of that risk:

overwrite	caught?
rebuilt (g_nodes, g_edges) with a new head	yes — graph_hash mismatch, hard refusal
same graph, refit g_head.npz / g_track.npz with different values	no — loads silently, results move, no local commit explains it

scripts/capture_golden.py records a SHA-256 of the four artifact files it read into tests/golden/parity.json for exactly this reason. Do not delete or rebuild artifacts from this repo without first checking whether another checkout points at the same data / outputs target.

---

5.7 Other files written

Not part of the load path, listed so they are not mistaken for one:

file	written by	read by
<run>/config.json, manifest.json, metrics.json, log.txt	onf.graph.report.StageLogger	humans; config.json is the run-reproduction record
<root>/<suite>/latest symlink (or latest.txt fallback)	StageLogger	Paths.graph resolves the symlink only — nothing reads latest.txt
trace_w<wid>_ep<n>.npz	onf.sentinel.sentinel.Sentinel._flush, only when SN_GRAPH_TRACE_DIR is set	debugging
tests/golden/parity.json	scripts/capture_golden.py	tests/test_parity.py

---

5.8 Easy to get wrong

1. Assuming a foreign head or kernel will fail loudly. It will not. Shapes match, values are finite, the answer is confidently wrong. graph_hash is the only thing between that and a moving arm — §5.2.
2. Treating a missing graph_hash key as "old file, probably fine". Both loaders map absence to mismatch and refuse. That is deliberate.
3. Rebuilding artifacts without checking the symlink target. A same-shaped refit passes the hash check and silently changes results — §5.6.
4. Reading sentinel_artifacts.json as a held-out guarantee against the head. It is not, and it says so in its own head_training_disjointness field — §5.3.
5. Deleting q_manifold*.npz from data/. Dead in this repo's code; data/ is shared with others that may still want them.
6. Expecting a missing g_track.npz to raise. GraphTracker.load falls back to unfit defaults and runs — a sentinel whose kernel is uniform, which is not what the results table was measured with.

---

5.9 Constants

constant	value	CHOSEN / DERIVED	what breaks
NODES_NPZ	g_nodes.npz	CHOSEN	filename contract; every loader resolves through it
EDGES_NPZ	g_edges.npz	CHOSEN	as above
HEAD_NPZ	g_head.npz	CHOSEN	default head_npz= argument of both load classmethods
TRACK_NPZ	g_track.npz	CHOSEN	_resolve_track_npz's directory-to-file expansion
ArtifactSchemas.HEAD.hash_stamped	True	CHOSEN	disabling it re-opens the foreign-head failure
ArtifactSchemas.TRACK.hash_stamped	True	CHOSEN	disabling it re-opens the foreign-kernel failure
_HASH_NODE_FIELDS	q, qdot, owner, task_id, stage, t_idx, phase	CHOSEN	dropping a field lets two distinguishable graphs collide
_HASH_EDGE_FIELDS	src, dst, rel, indptr	CHOSEN	as above
HELD_OUT_STRIDE	5	CHOSEN — 100/500 on long	shrinks the kernel fit set or the probe set
FIT_CLEAN_STRIDE	3	CHOSEN — wall-clock bound only	fit-set size, not composition
N_DRIFT_SEQ / N_CROSS_STRAND_SEQ	80 / 80	CHOSEN	the regimes that make beta identifiable at all
DRIFT_SEQ_SEED	0	CHOSEN	reproducibility of the sampled sequences

---

Next: 06-config-reference.md — every constant and environment variable, with its default and its provenance.