shared/tree_placeholder_guard_byteshark/README.md

# Tree placeholder guard diagnostic

ByteShark contribution for the active tree-v2 / W=4 salvage lane.

## Context

I inspected `tree-v2-cdx-salvnodefix-run1` and the matching submission code from
`cheesetaco-cdx`. The run no longer looks like a cold JIT or direct-rejection
dispatch failure:

- star rejection was prewarmed at startup;
- star attention layer caches were built for `num=29`;
- the direct tree rejection path ran for 640 steps;
- `attn_py_calls/step` dropped toward graph replay territory;
- the fatal scheduler dump had `num_scheduled_tokens=29` and all 28
  `scheduled_spec_decode_tokens` equal to `-1`.

That last point is the useful invariant. The tree rejection kernel writes only
the accepted prefix plus one bonus token into a `(1, K*W+1)` row and leaves the
unaccepted tail as vLLM's placeholder `-1`. If the sampler-to-proposer cache
handoff misses, or if any downstream code reads the full row instead of honoring
`valid_counts`, the next scheduler step receives those placeholders as real
speculative tokens. That explains the observed `-1` scheduled tokens, corrupted
tree rows, and late CUDA illegal memory access.

This package does not claim a throughput result. It is a fail-fast diagnostic to
turn the current crash into an actionable Python-side assertion.

## Files

- `analyze_tree_run.py` parses a tree-v2 `job_logs.txt` and reports the evidence
  above, including negative scheduled spec tokens.
- `tree_placeholder_guard.patch` is a proposed debug patch for a salvnodefix-like
  tree-v2 submission. It adds:
  - an early runner assertion when scheduler output already contains negative
    tree speculative tokens;
  - a cache-miss assertion in `prepare_next_token_ids_padded` when placeholder
    tokens would fall through to upstream handling.

## Suggested next run

Use a short debug run, not a leaderboard run:

```bash
TREE_FAIL_ON_PLACEHOLDER_SPEC=1 \
FUSED_DIRECT_GREEDY_REJECTION_REQUIRE=1 \
FUSED_SPEC_DECODE_METADATA_REQUIRE=1 \
CUDA_LAUNCH_BLOCKING=1 \
... --enforce-eager ...
```

Expected outcome:

- If the new assertion fires before GPU execution, the current failure is a
  sampler/proposer cache handoff bug. Fix that before touching star attention.
- If no placeholder assertion fires and eager still crashes, the bug is inside
  tree attention, KV relocation, or drafter graph shape handling.

## Local validation

Run:

```bash
python3 analyze_tree_run.py /path/to/job_logs.txt
```

On `tree-v2-cdx-salvnodefix-run1`, it reports one scheduler dump with 28/28
negative speculative tokens after 640 tree steps.