**Title:** Per-session file-modification‑time cache for auto‑revert

**Background**
The editor’s session system already warm‑swaps per‑session state (file explorer, LSP manager, panel IDs) when the active session changes.  The auto‑revert feature uses an in‑memory `file_mod_times` cache (keyed by file path) to detect external changes; however, that cache is currently global on `Editor` and is not swapped on session switch.

**Problem**
After switching to a different project root, the active session inherits the previous session’s mtime cache.  This can cause:
- Files belonging to the *inactive* session to trigger auto‑revert prompts because their old timestamps are still present in the cache.
- Files in the newly activated session to miss auto‑revert detection until the polling logic refills the cache from scratch.

The user’s mental model is that an inactive session is paused — its files should not auto‑revert until the user dives back in.

**Desired Behavior**
Each `Session` must own a private snapshot of the file‑modification‑time cache (`HashMap<PathBuf, SystemTime>`).  When switching away from a session:
- The active cache is moved into that session’s stash, leaving the active cache empty.

When switching to a session:
- That session’s previously stashed cache is moved back to the active slot.
- A freshly created session (or one never actvated) starts with an empty cache, so auto‑revert begins from a cold start after its first dive-in.

This guarantees:
1. Auto‑revert polls only the active session’s files.
2. Returning to a session presents the user with the same mtime state that existed when they left, immediately reflecting any external changes that occurred while away.

**Implementation Outline**
- Add a `file_mod_times_stash: HashMap<PathBuf, SystemTime>` field to the `Session` struct (starts empty for a new session).
- In `set_active_session`:
  - Take the current `Editor.file_mod_times` (via `std::mem::take`) and store it in the outgoing session’s `file_mod_times_stash`.
  - Restore the incoming session’s `file_mod_times_stash` into `Editor.file_mod_times` (again via `std::mem::take`), leaving the incoming stash empty.
- No changes are needed to the auto‑revert polling code – it only reads/modifies the active `file_mod_times` map, which is now correctly per‑session.

**Acceptance Criteria**
- Switching between sessions preserves each session’s mtime cache losslessly.
- No stale mtime entries from an inactive session interfere with auto‑revert prompts.
- A session that has never been active (or is fresh) starts with an empty cache upon first activation.

feat(core): warm-swap file_mod_times across sessions


**Bug:** Deleting a custom keybinding in the keybinding editor (`delete_selected`) marks it for removal, but `save_keybinding_editor_changes` never processes those removals. The deleted binding reappears after saving and reopening the editor.

**Expected behavior:**
1. After saving, config keybindings that match a deleted binding are permanently removed.
2. If a binding was added (`pending_adds`) in the same session and then deleted, it must be removed from `pending_adds` rather than tracked as a removal against the original config (it never existed on disk yet).
3. The removal tracking must be resistant to index shifts: identifiers should use the binding’s own data (action, key, modifiers, when), not list indices that become stale after edits.

**Implementation notes:**
- `pending_removes` should store `Keybinding` values (the config-level representation), not `usize` indices.
- During `delete_selected`, convert the selected `ResolvedBinding` to a `Keybinding`. If an identical binding exists in `pending_adds`, remove that pending add; otherwise, record the `Keybinding` in `pending_removes`.
- `save_keybinding_editor_changes` must first retain only config keybindings that do **not** match any entry in `pending_removes` (using field equality: action, key, modifiers, when), then append the new bindings from `pending_adds` as before.

Fix keybinding editor delete not persisting (pending_removes ignored)


**Feature Request: Prioritize exact matches in Open File fuzzy search**

When using the Open File dialog (Ctrl+O) and typing a filter, the ordering of results should reflect match quality more intuitively. Currently a substring match like `freshness` can appear above an exact match like `fresh` because only the base fuzzy score is considered.

**Desired behavior**
- An exact filename match (query equals the whole filename, case‑insensitively) always appears first.
- Next, an exact basename match: the query matches the filename up to an extension separator (`.`), a hyphen (`-`), underscore (`_`), or space (` `). For instance, typing `fresh` should rank `fresh.rs` and `fresh-editor` immediately after the exact match `fresh`.
- All other substring matches (e.g., `freshness`) appear after the matches above.
- The existing fuzzy scoring for consecutive letters, word boundaries, and start‑of‑string should still apply, but the hard bonus for exact/basename matches must override it so that the categories are strictly ordered.

**Acceptance criteria**
1. In a directory containing files: `fresh`, `fresh.rs`, `fresh-editor`, `freshness`, and opening the Open File dialog, typing `fresh` displays the entries in the order:
   - `fresh`
   - `fresh.rs`
   - `fresh-editor`
   - `freshness`
   (Or `fresh.rs` and `fresh-editor` may swap among themselves, but both must appear before `freshness`.)
2. The same ordering holds when the query is `config` and files are `config`, `config.rs`, `config_manager.rs`.
3. Case differences must not affect the ordering: `Fresh` vs `fresh` should be treated identically.
4. Non‑matching files are filtered out as before; no regression in filtering correctness.
5. The behavior is confined to the fuzzy matching used by the Open File dialog; file sorting other than by match score (e.g., by name/date columns) is not affected.

**Edge cases**
- Query equals the target exactly except for trailing extension (e.g., `fresh` → `fresh.rs`): should be treated as exact basename match, not substring.
- Query matches up to a separator character but the separator is part of the target filename in a way that is not an actual word boundary? The current proposal only considers `.`, `-`, `_`, and space as separators; other characters (e.g., digits) are treated as part of the word. This is acceptable.
- Query length equals target length but is not an exact match (e.g., `frezh` vs `fresh`): should not receive the exact‑match bonus; normal fuzzy score applies.

**Non‑goals**
- This does not change the fuzzy matching algorithm for other UI surfaces (Command Palette, etc.) unless they intentionally share the same `fuzzy_match` function. If they do, the same ordering improvements naturally apply.

feat: prioritize exact matches in Open File fuzzy search


**Title:** Restoring workspace after closing with a transient panel (e.g. *Search/Replace*) leaves a ghost split with duplicated file content

**Environment:** Fresh editor (all platforms), current workspace persistence

**Severity:** Medium – confuses users and clutters the layout on restart

---

### Steps to reproduce

1. Open any file (e.g., `foo.txt`).
2. Open the **Search/Replace** panel via the command palette (`Ctrl+P` → "Search/Replace") so that it appears in a split pane.
3. Close the editor normally (e.g., `Ctrl+Q` or menu → Quit) – let it save the workspace.
4. Reopen the editor in the same project directory. The workspace should be restored automatically.

### Observed behavior

- The editor restores a split layout with **two panes**.
- The *Search/Replace* panel is gone (as expected – it's transient), but the extra pane now shows **`foo.txt` again**, i.e. the same file that was open in the main pane.
- Effectively, the user ends up with a duplicate file view that they never explicitly created.

### Expected behavior

- The transient panel should leave no trace in the restored layout.
- After restart, there should be **exactly one pane** showing `foo.txt` – the same layout as if the panel had never been opened.
- No split or tab originating from a transient virtual buffer should be persisted.

### Why this matters

Transient virtual buffers like `*Search/Replace*`, lint results, or plugin panels are created on demand and can’t be rebuilt from disk. Persisting the split that contained them leads to an orphan pane that silently picks up whatever file happens to be available – usually a duplicate of the active buffer. This is confusing, visually noisy, and breaks the trust that the workspace state accurately reflects the user’s intentional layout.

### Root cause (from analysis)

During workspace save, `serialize_split_node` includes every leaf in the split tree, even those whose buffer is virtual (`BufferMetadata::is_virtual() == true`). When the workspace is later restored, the virtual buffer cannot be reconstructed, so the leaf ends up empty or re-attached to an existing real buffer, producing the duplicate pane.

The same problem can occur with **any** virtual buffer created by the core editor or plugins, not just the search panel.

### Required fix

The workspace serialization logic must:

- **Exclude** leaves (and entire subtrees) that only contain transient virtual buffers.
- When a `Split` node has one virtual child and one real child, **collapse the split** by hoisting the real child into the place of the whole `Split` node, so the saved layout does not contain an extra split slot.
- When both children of a `Split` are virtual-only, drop the entire `Split` node (nothing to persist).
- As a safety net, if the entire root split tree consists solely of virtual buffers, fall back to a minimal empty `[No Name]` leaf so the restored workspace is still valid.

This logic should **only** affect split-node serialization (layout persistence). The open-tabs/inactive-split-state serialization already skips virtual buffers and does not need changes.

### Non-obvious edge case: nested splits

If a virtual buffer sits inside a deeply nested split tree (e.g., `Split(Vertical, a.text, Split(Horizontal, *Search/Replace*, b.text))`), pruning must bubble up correctly so the final saved structure is simply `a.text` and `b.text` with a vertical split, without a leftover interior `Horizontal` split that held only one real child. The collapse logic must apply recursively upwards to the root.

### Additional notes

- This is a workspace-persistence bug; it does not affect the in-memory editor state while it’s running.
- The fix should be compatible with existing workspace files – a workspace that already contains orphan split data will be sanitised the next time it is saved, but the immediate restore after the fix may still show the old ghost split until a new save/restart cycle. That’s acceptable as long as new saves are clean.

fix(workspace): prune transient virtual-buffer splits from serialized layout


**Title:** Scrollbar thumb jumps to top of click in the keybinding editor

**Description:**
In the keybinding editor modal, the vertical scrollbar on the right side of the bindings table does not behave like a standard GUI scrollbar when clicked. Currently, pressing the left mouse button anywhere on the scrollbar (including directly on the thumb itself) causes the viewport to immediately scroll so that the *top* of the thumb aligns with the cursor. This makes the thumb “snap” upward when the user clicks somewhere in the middle or bottom of the thumb, breaking normal drag expectations and making it difficult to perform small, precise scroll adjustments.

**Acceptance Criteria:**

1. **Click on the thumb preserves position**
   When the user presses the left mouse button on any part of the scrollbar thumb (the visible draggable indicator), the viewport must **not** scroll. The current scroll offset must remain unchanged.

2. **Dragging the thumb follows the cursor correctly**
   When the user presses on the thumb and then drags the mouse, the viewport should scroll proportionally to the cursor movement, anchored to the exact point on the thumb where the press occurred. For example, if the user presses on the lower third of the thumb and drags downward, the thumb should follow the cursor smoothly, keeping that same relative point under the cursor.

3. **Click-release without movement is a no‑op**
   Pressing on the thumb and releasing the mouse button without any cursor movement must leave the viewport exactly as it was before the click.

4. **Click on the track still works as before**
   Pressing on the scrollbar track *outside* the thumb (above or below the thumb) should still immediately jump the viewport, centering the thumb on the clicked row. This existing behaviour must be preserved.

5. **Scroll range / clamping handled correctly**
   Dragging the thumb to the extremities must not allow the scroll offset to go out of bounds (below 0 or above the maximum). Content that fits entirely within the viewport must not cause any scroll changes when interacting with the scrollbar.

**Scope:** This applies to the keybinding editor modal only; other scrollable panels that use the same scrollbar widget may benefit from the corrected behavior, but this request is specifically about the keybinding editor’s scrollbar.

**Why this matters:** The current behaviour violates the common GUI convention where clicking on the thumb itself does not cause an immediate jump—only dragging or clicking the track does. Fixing this will make the keybinding editor feel more natural and responsive, especially for users with many bindings who need to scroll through the list accurately.

fix(keybinding-editor): preserve thumb-relative click on scrollbar drag


**Title:** Position history navigation (Alt+Left / Alt+Right) does not update the visible buffer content

**Severity:** High – basic navigation feature is unusable

**Steps to reproduce:**
1. Open or create two buffers with distinct, recognisable content (e.g., type “FIRST FILE” in buffer 1 and “SECOND FILE” in buffer 2).
2. Move the cursor around in one buffer so that a history entry is recorded.
3. Switch to the other buffer (Ctrl+N, or focus via tab).
4. Press `Alt+Left` to navigate back to the previous buffer.
5. Observe the editor display.

**What happens:**
- The tab indicator changes to highlight the target buffer’s tab.
- The main editing area continues to show the text of the buffer that was active *before* the navigation.
- The cursor remains at whatever position it had in the (still displayed) old buffer.

**What I expect:**
- The visible buffer content in the editing area should switch to the actual text of the target buffer.
- The cursor should be placed at the position that was saved in the position history for that buffer.

**Impact:**
This makes Alt+Left/Alt+Right effectively broken – you can “travel” through the history in the tab bar but you never see the content of the buffer you navigated to. The only way to see the correct content is to manually click a tab, which defeats the purpose of the position history navigation.

**Additional context:**
The issue occurs every time with any number of buffers. It doesn’t matter whether the history entry was created by cursor movement, page scrolling, or opening a new file. The problem appears to be that the split view is never told to display the newly activated buffer when navigating history; only the editor’s internal active buffer id is updated, but the rendering system still renders the old buffer.

Fix position history navigation to update visible buffer content


**Title:** Validate chunked recovery reconstruction and tidy related debug output

**Background**
The recovery system now supports chunked recovery: when a buffer is backed by a disk file, only the diff chunks (byte-range replacements) are persisted in the recovery metadata, rather than the full unsaved content. The reconstruction logic in `RecoveryStorage::reconstruct_from_chunks` re-applies these chunks on top of the original file to recover the full buffer state. This path has not yet been exercised end-to-end.

Additionally, the file-watcher handler currently logs every external change event for non-open files at `debug` level, which produces excessive noise in normal usage. It should be downgraded to `trace`.

Finally, `RecoveryStorage::with_dir()` is gated with `#[cfg(test)]`, preventing its use from integration/e2e test suites outside the crate. It should be available unconditionally for such tests.

**Request**

1. **Add end-to-end coverage for chunked reconstruction** using `RecoveryStorage` with a temporary directory. The tests should cover:
   - **Basic reconstruction**: multiple non-overlapping byte-range replacements on a known original file, verifying the final reconstructed content matches the expected result.
   - **Insertion**: a chunk that replaces zero bytes at a position (i.e. inserts new bytes) – the reconstruction must grow the original content correctly.
   - **Deletion**: a chunk that replaces a range with zero bytes (i.e. deletes bytes) – the reconstruction must shrink the content correctly.
   - **Size mismatch error**: when the original file on disk has a different length than the `original_file_size` stored in the chunked metadata, `reconstruct_from_chunks` must return an error indicating a size mismatch.

   Use `RecoveryStorage::with_dir()`, `save_chunked_recovery`, and `reconstruct_from_chunks` with temporary files.

2. **Change the log level** of the message `"Ignoring file change event for non-open file: …"` in the async file change handler from `debug` to `trace`. This event is expected and not actionable; it should not appear in normal debug logs.

3. **Remove the `#[cfg(test)]` gate** from `RecoveryStorage::with_dir()` so integration tests can construct a recovery storage pointed at an arbitrary directory.

**Acceptance Criteria**
- Reconstruction tests pass with both insertions and deletions.
- A test with an intentionally mismatched original file size fails reconstruction cleanly with a “size mismatch” error.
- The file-watcher message no longer appears at `debug` level.
- `RecoveryStorage::with_dir()` is public and usable from external integration tests.

Add e2e tests for chunked recovery and reduce file watcher log noise


Shift+Backspace currently has no effect because the keybinding resolver looks for an exact match on `(Backspace, SHIFT)`, which is not bound anywhere. Users holding Shift (for example right after typing a capital letter) expect it to behave exactly like plain Backspace and delete a character backward.

**Required behavior**
1. **Keybinding resolution:** In the resolver’s key normalization, strip the `SHIFT` modifier from `Backspace` so that it matches existing `Backspace` bindings. This mirrors the normalization already done for `BackTab`. Shift+Backspace should resolve to the same action as plain Backspace in every context where Backspace has a binding (`Normal`, `Prompt`, `FileExplorer`, etc.).
2. **Completion popups:** The completion popup handlers check modifiers directly before acting on Backspace; extend those guards to also accept `SHIFT` alone, so that Shift+Backspace removes the last filter character in both `handle_completion_input` and `handle_completion_input_with_popup`.

No new actions or keybindings should be introduced; the change is purely about making the editor treat Shift+Backspace as logically identical to Backspace throughout all input handling.

fix(input): treat Shift+Backspace the same as Backspace


Write a repository-specific implementation request that could plausibly have been written before this patch existed. Frame it as a maintainers' backlog ticket with concise repository context and a concrete ask.

nvmf: handle PRACT bit in SGL length validation


**Subject:** `spdk_fd_group_nest()` fails with assertion when nesting depth > 1

**Observed Behavior**
Calling `spdk_fd_group_nest()` on a child that is already nested under another parent triggers an assertion failure (`assert(false)`) and returns `-ENOTSUP`.
Only a two-level hierarchy (parent–child) is allowed; deeper trees (e.g. root → child1 → child2) are not supported.

**Expected Behavior**
`spdk_fd_group_nest()` should allow arbitrary nesting depth. All file descriptors registered in any group within a nesting tree should be added to the **root** group’s epoll instance.
Intermediate groups should have `num_fds == 0`; the root group must correctly track the total number of fds across the entire tree.

Operational semantics across the tree:
* `spdk_fd_group_add()` / `_add_ext()` on any nested group must register the fd with the root epoll and increment the root’s `num_fds`.
* `spdk_fd_group_remove()` must remove from the root epoll and decrement the root’s count.
* `spdk_fd_group_event_modify()` must update the event mask on the root epoll.
* `spdk_fd_group_nest()` must move all fds from the child’s own epoll (and any children it already has) to the root’s epoll, updating counts accordingly. Error recovery must be attempted so that a partial migration does not leave the tree in a broken state.
* `spdk_fd_group_unnest()` must move the subtree’s fds back from the root’s epoll to the child’s epoll, re-establishing that branch as an independent group, again with proper error recovery.

**Impact**
Users cannot build deeply nested I/O polling hierarchies (e.g., a tree of subsystems where each node may aggregate event sources from lower nodes). This forces awkward workarounds and prevents clean modular composition of fd groups.


---
title: Make band relocation queue membership explicit with a state machine in `ftl_reloc`

**Background**
The FTL relocation module (`lib/ftl/ftl_reloc.c`) currently uses a single boolean `active` flag to track whether a band is being actively relocated. While each band already belongs to one of three queues (pending, active, high-priority), the code determines queue membership implicitly by checking the flag and scanning queues. This has led to fragile transitions, especially when a band changes priority or receives an ANM (Additional New Modifications) event while relocation is in progress.

**Problem**
- It is impossible to distinguish between “not in any queue” (fully relocated / empty band) and “in the pending queue” without inspecting the queues.
- High-priority bands need to acquire the LBA map immediately and remain in the high-priority queue until fully relocated, ignoring ANM events. With only a boolean, the code cannot easily express that a band is in the priority queue vs. the active queue.
- When an ANM event occurs for a non-priority band that is currently being relocated, the band must be returned to the pending queue for re-evaluation. Without explicit state, this move is error-prone and may leave the band unqueued or in the wrong queue.
- The existing helper `ftl_reloc_add_active_queue` blurs the separation between queue insertion and resource preparation, leading to duplicated logic.

**Expected behavior**
Introduce an explicit state for each band relocation (`ftl_band_reloc::state`) with the following states:
- **INACTIVE** – the band is not in any relocation queue (fully relocated, empty, or not yet added).
- **PENDING** – the band is waiting in `pending_queue`.
- **ACTIVE** – the band is being relocated with normal priority and is in `active_queue`.
- **HIGH_PRIO** – the band is being relocated with high priority and is in `prio_queue`.

**Queue and resource rules**
1. When a band is added first time with normal priority, set state to PENDING and insert into pending_queue.
2. When the reloc loop (in `ftl_reloc()`) picks a band from pending_queue, call `ftl_reloc_prep()` to acquire its LBA map, then move it to ACTIVE state and insert at the head of active_queue.
3. If a band receives a high-priority request (`prio = 1`):
   - If it is currently ACTIVE, remove from active_queue, set state to HIGH_PRIO, and insert at tail of prio_queue. No additional LBA map acquisition is needed because the map is already held.
   - If it is PENDING, remove from pending_queue, call `ftl_reloc_prep()` to acquire the LBA map, set state to HIGH_PRIO, and insert into prio_queue.
   - If it is INACTIVE, do the same: call prep, set state, insert into prio_queue.
4. When a band in ACTIVE or HIGH_PRIO state finishes all round of `ftl_process_reloc()` (i.e., `ftl_reloc_done()` is true), call `ftl_reloc_release()`. In `ftl_reloc_release`:
   - For HIGH_PRIO state: assert the band is completely empty (num_lbks == 0), remove from prio_queue, clear band's `high_prio` flag, set state to INACTIVE.
   - For ACTIVE state: remove from active_queue, set state to INACTIVE. If the band still has pending blocks (`num_lbks != 0`), it means an ANM event arrived during active relocation; in that case set state back to PENDING and insert at tail of pending_queue so it will be reprocessed later.
5. When the relocation module is halted (shutdown), only bands in ACTIVE or HIGH_PRIO state need to drain their move queues; use the state instead of the boolean to decide.
6. After a relocation is fully completed and the band becomes empty, set state to INACTIVE.

**Constraints**
- The existing `defrag` flag and defragmentation behavior must remain unchanged.
- `ftl_reloc_prep` must still increment `reloc->num_active` and prepare move resources, regardless of whether the band becomes ACTIVE or HIGH_PRIO. The state transition (PENDING → ACTIVE or ANY → HIGH_PRIO) should happen after calling `ftl_reloc_prep` where needed.
- All assertions that currently check `breloc->active` must be updated to check the new state field (e.g., INACTIVE after cleanup, ACTIVE while processing the active queue).
- The `ftl_reloc_add_active_queue` helper should be removed, and its logic inlined into the caller with proper state management.
- No functional regression in how ANM events cause a band to stay in or return to the pending queue for normal-priority relocations.

**Goal**
Provide a clear, maintainable state machine for band relocation that eliminates ambiguity, reduces duplication, and prevents bugs related to queue membership and resource acquisition.

lib/ftl: Keep state of single band relocation


As a user of the SPDK JSON‑RPC management API, I need the ability to retrieve the current configuration of a single, named subsystem in JSON format. This allows me to inspect or automate management of individual subsystems without dumping the entire application configuration.

**Acceptance Criteria:**

1. **New RPC method `get_subsystem_config`:**
   * Accepts a `name` parameter (string) specifying the target subsystem.
   * If a subsystem with that name exists, the method returns a JSON representation of that subsystem’s current configuration, as produced by the subsystem’s own serialization logic.
   * If the named subsystem exists but has not registered a JSON configuration output handler, the method returns JSON `null` for the result.
   * If no subsystem with that name is registered, the method responds with an `INVALID_PARAMS` JSON‑RPC error, and the error message includes the requested subsystem name (e.g. `"Subsystem 'xyz' not found"`).

2. **Opt‑in per‑subsystem JSON serializer:**
   * The subsystem registration mechanism must accept an optional callback (`write_config_json`) that takes a JSON write context and writes the subsystem’s configuration directly to it.
   * This callback is invoked only by the new RPC method; other existing configuration pathways (e.g., legacy `FILE *` output) remain unchanged.

3. **Python RPC binding:**
   * The Python helper library must expose a corresponding `get_subsystem_config(name)` function, which forwards the call and prints the returned JSON value.

**External Constraints:**
* The new RPC method must not affect the existing `get_subsystems` method or the existing subsystem initialization/teardown flow.
* The implementation must follow the existing SPDK patterns for subsystem definition, RPC registration, and JSON formatting.

subsystem: add per module configuration dump


**Title:** Replace TAILQ-based timed poller ordering with a Red-Black tree for O(log n) insertion and removal
**Component:** `lib/thread` (internal thread poller infrastructure)
**Priority:** Medium – performance improvement for workloads with many periodic pollers

### Problem
The timed pollers on each SPDK thread are currently kept in a TAILQ sorted by `next_run_tick`. Inserting a poller into this list (`poller_insert_timer`) scans the list in reverse to find the correct insertion point, giving O(n) complexity. This becomes a measurable bottleneck when a thread manages a large number of timed pollers.

SPDK already ships a Red-Black tree implementation (`spdk/tree.h`) that provides `RB_INSERT`, `RB_REMOVE`, `RB_FOREACH`, etc. Switching to a tree will reduce insertion/removal to O(log n) while keeping the pollers ordered by expiration tick.

### Required Behavior
1. **Data structure change:**
   - Replace `TAILQ_HEAD(timed_pollers_head, spdk_poller)` with `RB_HEAD(timed_pollers_tree, spdk_poller)`.
   - Replace the `TAILQ_ENTRY` linkage in `struct spdk_poller` with an `RB_ENTRY` node.

2. **Key and ordering:**
   - Key each poller by its `next_run_tick` (absolute time when it should fire next).
   - The tree must allow *duplicate keys*: multiple pollers may share the exact same `next_run_tick`, and all of them must remain schedulable. The standard RB macros treat equal keys as a collision and `RB_INSERT` would return a pointer to the existing element instead of inserting the new one. To support duplicates, the compare function should **never return 0** when the ticks are equal (e.g., return 1) so that the new node is always inserted as a right child. This ensures duplicates are kept in the tree and traversed.

3. **Operations to update:**
   - **Register / insert timer** (`poller_insert_timer`): Insert into tree. Only update the thread’s `first_timed_poller` cache when the tree was previously empty *or* the new poller’s tick is strictly less than the current cached leftmost poller. Because duplicates are inserted on the right, the leftmost element (minimum tick) does not change when an equal tick is inserted.
   - **Unregister / remove timer** (`poller_remove_timer`, and inline removal during polling): Use `RB_REMOVE`. When the removed poller was the cached `first_timed_poller`, recompute the new leftmost via `RB_MIN`.
   - **Polling loop** (`thread_poll`): Change the timed poller processing from `TAILQ_FOREACH_SAFE` and `TAILQ_REMOVE` to `RB_FOREACH_SAFE` and `RB_REMOVE`, while preserving the existing logic for `first_timed_poller` cache maintenance (only update if the current poller is still the leftmost).
   - **Thread exit / destruction**: Walk and remove all timed pollers using `RB_FOREACH_SAFE` and `RB_REMOVE`.
   - **Pause / resume**: These paths move pollers between lists/trees. Ensure they use the new tree interface consistently.
   - **Interrupt mode toggling** (`spdk_thread_set_interrupt_mode`): Traverse timed pollers with `RB_FOREACH_SAFE`.

4. **Public API and iteration helpers:**
   - `spdk_thread_get_first_timed_poller` should return `RB_MIN`.
   - `spdk_thread_get_next_timed_poller` should return `RB_NEXT`.
   - `thread_has_unpaused_pollers` must use `RB_EMPTY` on the timed pollers tree.

5. **Assertions and safety:**
   - Because SPDK will be using RB tree insertion here for the first time, check that `RB_INSERT` returns NULL (ensuring the new node was actually inserted); add an `assert` for safety.
   - `RB_REMOVE` should similarly validate that the element was present (though that is less critical). The patch may later remove the temporary variable for `RB_REMOVE` to keep the diff clean, but the initial change should at minimum verify correct removal.

6. **Performance invariants:**
   - No regression for small poller counts.
   - Insertion and removal must be O(log n) instead of O(n).
   - The `first_timed_poller` cache must remain correct and efficient; avoid unnecessary `RB_MIN` calls when inserting a non-leftmost element.

### Notes
- All operations happen under the owning thread’s context; no additional locking is required.
- The change should not alter externally observable behavior – poller execution order among equal `next_run_tick` values may differ from the TAILQ order, but that is permissible (the contract only guarantees expiration-based ordering, not FIFO among identical ticks).

thread: Replace TAILQ by Red Black tree for timed pollers


The NVMe driver’s admin command completion path must avoid a use-after-free when a command fails.

### Acceptance Criteria

1. **No copy on error**
   When an admin command submitted via `nvme_allocate_request_user_copy` completes asynchronously with a failed status (any SCT / SC combination that is not success), the completion callback **must not** copy payload data from the internal DMA buffer into the user-supplied buffer.
   The user buffer must be left **unmodified**.

2. **Successful path unchanged**
   For completions with a successful status, the existing copy-back behavior must be preserved: if a user buffer was provided and the payload size is non‑zero, the data is copied as before.

3. **Guard order**
   The error check must be placed before the copy logic so that a failure status short‑circuits the copy, regardless of the payload size or buffer pointer.

4. **Safety constraint**
   This prevents a use-after-free scenario where the caller (e.g., `spdk_opal_dev_construct`) frees the user buffer after detecting a command failure via `nvme_wait_for_adminq_completion`, while the eventual completion callback would otherwise try to write into that freed memory.

The fix is expected to resolve issue #3861.

lib/nvme: don't copy to user buffer if req failed


We need to enable dynamic management of initiator group memberships. Currently, initiator names and netmasks are stored as arrays that are set once at creation, which makes it difficult to add or remove entries later without re-creating the group, and it is not straightforward to detect duplicates within a batch.

**Desired behavior:**
- After an initiator group exists, it should be possible to **add a single initiator name** to the group. If the name is already present, the operation must fail with `-EEXIST`. If the group already holds the system‑defined maximum number of initiators (`MAX_INITIATOR`), it must fail with `-EPERM`.
- Similarly, it should be possible to **delete a single initiator name** from the group. If the name is not found, it must fail with `-ENOENT`.
- The same capabilities must exist for **netmasks** (add/delete with `-EEXIST`, `-EPERM`, and `-ENOENT`), using the maximum netmask count (`MAX_NETMASK`).
- The existing bulk‑add operations (`spdk_iscsi_init_grp_add_initiators` / `spdk_iscsi_init_grp_add_netmasks`) must be implemented in terms of the per‑item add functions. This guarantees that duplicates **within the same batch** are rejected, and that if any single item fails (duplicate, limit, or allocation), the entire batch is rolled back—no partially added entries remain.
- When an initiator group is destroyed, all associated initiator‑name and netmask structures and strings must be freed.
- All existing traversals (access‑control checks in `spdk_iscsi_tgt_node_access` and `spdk_iscsi_tgt_node_visible`, JSON output in the `get_initiator_groups` RPC, and configuration dump in `spdk_iscsi_config_dump_initiator_groups`) must continue to see initiator names and netmasks in **insertion order** (tail‑added order).
- The internal representation must efficiently support the above add/delete semantics and duplicate detection without changing the external interfaces for group creation or target mapping.

iscsi: manage IG's name and mask by linked list


Implement the missing `submit_process_request` callback in the RAID1 module so that rebuild operations are supported. The RAID bdev layer uses this callback to copy data from operational base bdevs to a target base bdev during rebuild.

Requirements:

- Register a `submit_process_request` handler in the RAID1 module struct.
- The handler must read the requested blocks from a healthy base bdev (using the existing read path) and then write that data to the target base bdev referenced in the process request.
- On read failure, complete the process request with an error.
- On write submission failure:
  - If the error is `-ENOMEM`, queue and retry the write asynchronously.
  - For other errors, complete the process request with the error.
- On successful write, complete the process request with success (status 0).


**Title:** `_nvme_qpair_build_hw_sgl_request` calls `next_sge_fn` after reaching `NVME_MAX_SGL_DESCRIPTORS`

**Background**
In `lib/nvme/nvme_qpair.c`, `_nvme_qpair_build_hw_sgl_request` fills a tracker’s SGL descriptor array by calling the request’s `next_sge_fn` callback for each segment. It must stop once `NVME_MAX_SGL_DESCRIPTORS` descriptors have been added.

**Problem**
The current loop fetches a segment, increments the descriptor count (`nseg`), appends the SGL descriptor, and *then* checks `if (nseg >= NVME_MAX_SGL_DESCRIPTORS)`. This means when the limit is hit the function has already called `next_sge_fn` one extra time, but it never uses the returned segment and immediately fails the request. This extra call can advance internal state of the SGL provider incorrectly and leads to a wasted callback.

**Expected Behavior**
The check must happen *before* invoking `next_sge_fn`. When `nseg` is already equal to `NVME_MAX_SGL_DESCRIPTORS` at the start of a loop iteration, the function should fail the request without calling `next_sge_fn` again.

**Acceptance Criteria**
- In the `while (remaining_transfer_len > 0)` loop, the check `nseg >= NVME_MAX_SGL_DESCRIPTORS` appears before the call to `req->payload.u.sgl.next_sge_fn(...)`.
- When the descriptor limit is reached, `_nvme_fail_request_bad_vtophys` is called and the loop exits immediately.
- No change in the logic for the single-segment fast path or other payload types.


**Title:** Use Asynchronous Probe API Internally in `spdk_nvme_probe` and `spdk_nvme_connect`

**Background**
The library already offers `spdk_nvme_probe_async()` and `spdk_nvme_probe_poll_async()` to allow callers to integrate NVMe controller initialization into their own event loops. However, the synchronous `spdk_nvme_probe()` still uses a separate blocking initialization path, and `spdk_nvme_connect()` uses a stack-allocated probe context that is incompatible with the async API’s expectations.

**Request**
Refactor `spdk_nvme_probe()` and `spdk_nvme_connect()` so they reuse the existing async probe primitives internally, eliminating the duplication and aligning the internal code paths.

**Concrete requirements**

1. `spdk_nvme_probe()` must:
   - Call `spdk_nvme_probe_async()` to obtain a probe context.
   - If `spdk_nvme_probe_async()` fails, log an error and return an appropriate error code (`-1`).
   - Enter a polling loop that calls `spdk_nvme_probe_poll_async()` repeatedly until it returns something other than `-EAGAIN`.
   - Return the final result of that loop. If one or more controller attach failures occur, the error must still be propagated as it was before.

2. `spdk_nvme_connect()` must:
   - Allocate the probe context on the heap (using `calloc`) so that it remains valid throughout the asynchronous initialization path.
   - Continue to use `spdk_nvme_probe_ctx_init` and `spdk_nvme_probe_internal` (with `direct_connect = true`) to set up the context and kick off discovery, but the context must be allocated dynamically.

3. Both functions must maintain their observable behavior:
   - `spdk_nvme_probe()` remains synchronous from the caller’s perspective—it returns only after all discovered controllers have been through the full initialization state machine and the attach callback has been invoked.
   - `spdk_nvme_connect()` still returns the fully initialized controller or `NULL` on error.

**Edge Cases / Constraints**
- The `nvme_init_controllers()` helper must be simplified to just run the polling loop on the passed-in context, removing the direct controller iteration logic.
- The shared driver’s `initialized` flag was previously set inside the old `nvme_init_controllers()`; after the refactoring it should be set at the appropriate point during the async initialization sequence (already handled by the async API).
- No changes to the public API signatures—`spdk_nvme_probe()` and `spdk_nvme_connect()` retain their current parameters and return types.

nvme: use asynchronous probe API in spdk_nvme_probe()


As a maintainer, I’d like to reduce unnecessary coupling between the FTL relocation module and the `ftl_io` subsystem.

**Background**
Internally, `lib/ftl/ftl_reloc.c` currently allocates `ftl_io` objects from the general I/O pool and uses them as entries in its own free and write queues. This mixes relocation bookkeeping with the I/O layer: the I/O object carries the data buffer, PPA, and callback indirectly, while the relocation module must reinitialize and reinterpret these fields. It also ties the lifetime of the buffer and I/O together, complicating resource cleanup.

**Ask**
- Introduce a dedicated, lightweight internal structure to represent a pending relocation request. This object should own the data buffer and the start PPA, know how many logical blocks it covers, and optionally hold a pointer to an `ftl_io` only while that I/O is active.
- Change the reloc free and write queues to hold pointers to these new move objects instead of `ftl_io` pointers.
- Allocate the data buffer directly in the relocation module—no longer use `ftl_io` to store it.
- When a relocation move needs to issue a read or write, create a temporary `ftl_io` for that I/O only, populating it with the move’s buffer, PPA, and callback. After the I/O completes, free the `ftl_io` and return the move object to the appropriate queue.
- Ensure that the write path can carry an LBA vector (`FTL_IO_VECTOR_LBA`). To support this, `ftl_io_init_internal` must allocate an LBA vector when the flag is set, and `_ftl_io_free` must free it.
- Keep the total number of move objects bounded; use a pre-allocated pool per band (size equal to the existing max queue depth) to avoid dynamic allocations during relocation.
- Maintain existing functional behavior: the reloc iteration, valid-LBK scanning, and error handling should be unchanged except for the internal representation.

**Outcome**
The relocation module no longer depends on `ftl_io` for queue management. I/O objects are used only for actual read/write submission, and the move structure provides clear ownership of per-request state. This improves separation of concerns and simplifies future changes to either the reloc or I/O layer.

lib/ftl: Stop using ftl_io for internal relocation logic


**Summary**
The NVMe-oF target uses a poll group per thread, but currently each subsystem holds a single IO channel per namespace. In a multi-threaded environment, each thread needs its own set of namespace IO channels to safely handle I/O without cross‑thread contention. We need to extend the poll group to maintain per‑subsystem, per‑thread IO channel storage so that future I/O processing can use those channels directly from the polling thread.

**Acceptance Criteria**

1. **Subsystem creation notification**
   When `spdk_nvmf_create_subsystem()` succeeds, every existing poll group (i.e., every thread’s poll group) must be notified and must allocate a per‑subsystem structure holding an IO channel for each namespace currently belonging to the subsystem.
   - Notification must be delivered on each poll group’s own thread context (via `spdk_for_each_channel`).
   - The per‑subsystem structure must index channels by namespace id.

2. **Subsystem deletion cleanup**
   When `spdk_nvmf_delete_subsystem()` is called, the reverse must happen: all poll groups must destroy their per‑subsystem data, release every held IO channel, and free all associated memory.
   - No memory leaks or dangling channel references may remain after the subsystem is deleted.

3. **New poll group initialisation**
   When a new poll group is created (i.e., a new thread starts serving the target), it must scan all currently active subsystems and create the same per‑subsystem structures with IO channels for every namespace.
   - This must be done during poll group creation (inside `spdk_nvmf_tgt_create_poll_group` or its equivalent).

4. **Poll group destruction**
   When a poll group is destroyed, it must release all per‑subsystem IO channels and free the associated memory.
   - No channels may be leaked if a poll group is destroyed while subsystems still exist.

5. **Resilience to rank changes**
   If a subsystem’s id is larger than the current allocated array size in a poll group, the poll group must grow its subsystem array to accommodate the new id.
   - The number of channels in the per‑subsystem structure must match the maximum nsid configured for that subsystem.

6. **Thread safety**
   Operations that add or remove subsystems must be serialised across threads through `spdk_for_each_channel` (or an equivalent one‑per‑thread broadcast) to ensure the poll group’s internal state is updated safely.
   - No direct cross‑thread pointer passing; each poll group’s update runs on its own thread.

7. **Scope limitation**
   At this stage, the newly created channels are **not yet used** for any I/O command processing. The only requirement is that the lifecycle of these per‑poll‑group channels is correctly tied to subsystem creation/deletion and poll group creation/destruction, without leaking resources or leaving stale pointers.

nvmf: Add a channel map to the poll group


**Title:** `nvme: crash in nvme_cb_complete_child when resubmitting a queued child request fails`

**Area:** `lib/nvme` – request submission and child completion

**Description:**

When a parent request with one or more child requests is submitted on an I/O qpair, the children are linked via `req->parent`. If the submission cannot be completed immediately (e.g., due to transport back-pressure), the request tree is placed on the qpair’s `queued_req` list. Later, during `spdk_nvme_qpair_process_completions`, the queued requests are resubmitted via `_nvme_qpair_submit_request`.

If that **resubmission fails** (specifically with `-ENXIO` or another error that triggers `nvme_qpair_manual_complete_request` / `nvme_cb_complete_child`), the error path in `_nvme_qpair_submit_request` currently does the following:

1. It calls `nvme_request_remove_child(req->parent, req)`, which clears `req->parent`.
2. It then calls `nvme_qpair_manual_complete_request(...)` for the child request, which eventually invokes `nvme_cb_complete_child` → `nvme_request_remove_child(parent, child)` **again**.

The second call dereferences the now‑`NULL` parent pointer, resulting in a crash.

**Expected behaviour:**

- Failing to resubmit a queued child request must not double‑remove the child from its parent.
- The child should be removed from the parent exactly once, through the standard completion callback path used for error reporting, without touching `req->parent` prematurely in the error label of `_nvme_qpair_submit_request`.

**Constraints / edge cases to respect:**

- Non‑queued requests (new, immediate submissions) should continue to be cleaned up directly in the error path, including removal from the parent if applicable.
- The fix must not break the counting of outstanding children for fused operations or any other logic that relies on `num_children` / `req->parent` membership.
- The change should keep the existing caller‑side completion trigger for queued requests unchanged (the `spdk_unlikely(req->queued)` branch).

The issue is reproducible when a transport error (`-ENXIO`) is injected on resubmission of a command tree (e.g. a two‑command fused operation) that was previously queued.

**Ask:** Rework the error handling in `_nvme_qpair_submit_request` so that queued child requests are not explicitly unlinked before they are completed, thereby avoiding the NULL‑pointer dereference in `nvme_request_remove_child`.

nvme: fix NULL pointer dereference in nvme_cb_complete_child


**Title**: Refactor iSCSI target node access control to eliminate redundant repetition

**Background**
The access control procedure for login in `spdk_iscsi_tgt_node_access()` follows a fixed decision table:

```
+------------------------------+
|iscsi name |netmask  |result  |
+------------------------------+
|denied     |-        |denied  |
+------------------------------+
|allowed    |allowed  |allowed |
+------------------------------+
|allowed    |denied   |next IG |
+------------------------------+
|not found  |-        |next IG |
+------------------------------+
```

The current implementation contains redundant loops and condition checks that make the logic hard to follow and obscure the above table. There is also unnecessary duplication when the same initiator group is associated with multiple portal groups in `spdk_iscsi_tgt_node_allow_iscsi_name()`.

**Request**
1. Refactor `spdk_iscsi_tgt_node_access()` to clearly implement the decision table, removing any repeated checks. The control flow should directly reflect:
   - If the initiator name is explicitly denied in the current initiator group, **deny** immediately.
   - If the name is explicitly allowed, check the netmask: if the netmask matches, **allow**; otherwise, move to the next initiator group.
   - If the name is not found in the current group, move to the next initiator group.
   - Default to **deny** when all groups are exhausted.

2. Refactor `spdk_iscsi_tgt_node_allow_iscsi_name()` so that it skips initiator groups that have already been checked (same `ig_tag`), avoiding repeated evaluation for multiple maps that reference the same initiator group. The function’s external semantics must be preserved.

3. Ensure the refactored code maintains **identical runtime behavior** to the current release. No change in access decisions is allowed. The change is purely a cleanup of internal structuring to improve maintainability and readability.

**Constraints**
- Must not alter any externally visible access control outcome.
- Must not introduce new dynamic allocations or performance regressions in the hot login path.
- Must handle all edge cases implied by the table (empty initiator lists, missing netmasks, etc.).

iscsi: Remove redundant repetition from ACL


```
Title: JSON-RPC server rejects requests with "params": null, breaking Rust clients

Observed problem:
- When a JSON-RPC request includes `"params": null` (i.e., the JSON literal null), the SPDK JSON-RPC server currently returns an error response (invalid params).
- This happens even though the called method takes no parameters; the request would succeed if `params` were omitted entirely.

Why this matters:
- Rust JSON-RPC libraries (e.g., those using serde) serialize the empty/unit value `()` as `null`. A method call with no parameters then becomes `"params": null`.
- This makes it impossible to use standard Rust JSON-RPC clients with the SPDK JSON-RPC server without special workarounds.

Expected behavior:
- The server should treat a JSON `null` value for `"params"` identically to an absent `"params"` member: the handler should receive a `NULL` pointer for `params`, and the request should proceed normally.
- If `"params"` is present and is not `null`, the existing validation (must be an array or object) should still apply.
- This improves interoperability with clients that naturally emit `"params": null` for parameterless calls.

Additional constraints:
- No changes to the behavior for requests with valid array/object `params` or with `params` omitted entirely.
- The change must not introduce memory errors or bypass the existing check that rejects scalar `params` values other than `null`.

Impact: Blocks adoption by projects that use Rust-based JSON-RPC clients against SPDK’s RPC API.
```

jsonrpc: tolerate null json value for parameters in json-rpc call


**Title:** `spdk_for_each_channel` may use freed I/O channel when iteration races with channel teardown
**Component:** `lib/util/io_channel.c`
**Type:** bug

### Problem
`spdk_for_each_channel` sends a message to every thread that has an I/O channel for a given `io_device`. While the message is queued on a thread, that thread's channel may be released via `spdk_put_io_channel` and freed. The current implementation stores a pointer to the channel in `struct call_channel.cur_ch` at send time and uses it unconditionally inside `_call_channel`, leading to a use-after-free.

### Expected behavior
When `_call_channel` runs on a target thread it must **re‑lookup** the channel for `io_device` under `g_devlist_mutex` using the thread’s channel list (`thread->io_channels`).
- If the channel is still present: call `fn(io_device, ch, ctx)` as usual.
- If the channel has been deleted (i.e. not found in the list): **skip** that thread entirely—do not invoke `fn`—and continue iteration to the next thread.

The same safe lookup must be performed when advancing to the next thread inside `_call_channel`, and no stale channel pointer may be stored in `struct call_channel`.

### Constraints
- No crashes, no dereference of freed memory, no use of a dangling `cur_ch` pointer.
- The `g_devlist_mutex` must be held across the traversal of `thread->io_channels` and any read of channel data.
- The completion callback (`cpl`) must be invoked exactly once after all threads have been examined, even if some threads no longer held the channel when the message executed.
- The `struct call_channel` should no longer carry a `cur_ch` field.
- The initial setup in `spdk_for_each_channel` must not store a channel pointer for the first thread; the lookup is deferred to the message handler.

### Relevant API
`spdk_for_each_channel(void *io_device, spdk_channel_msg fn, void *ctx, spdk_channel_for_each_cpl cpl)` in `lib/util/io_channel.c`.
Helper `_call_channel(void *ctx)` is the per‑thread message callback; its `ch_ctx` points to a `struct call_channel`.

lib/util: handle channel being removed during spdk_for_each_channel


We noticed that RAID1 write handling does not properly use mirror redundancy and can misreport status when a base device write fails. Specifically:

- The initial status for a write I/O is not explicitly set, so if all writes fail the overall status can be SUCCESS.
- A write failure on a base device does not mark the device as failed, so subsequent I/Os may still target a broken member.
- Missing base channels (e.g., a device that has not been fully configured) are treated as successful write completions.

We need to make RAID1 writes robust:

1. When a RAID1 write I/O starts, set its default completion status to `SPDK_BDEV_IO_STATUS_FAILED`.
2. In the write completion callback, if the base device write failed, call `raid_bdev_fail_base_bdev` on the corresponding `raid_base_bdev_info`.
3. For any base channel that is NULL (missing), treat it as a failed completion with `SPDK_BDEV_IO_STATUS_FAILED`.
4. The usual partial-completion logic (`raid_bdev_io_complete_part`) should then correctly upgrade the overall status to SUCCESS if at least one base device write succeeds, and leave it as FAILED only when all writes fail.

This gives us the expected mirror behavior: a write succeeds as long as at least one member succeeds, and a failing member is taken offline.


Add support for writing expectation results to GCP Cloud Logging when the audit sink is configured as GCPLOG. Currently expectation reports are only output through the SQL-based audit table. Mirror the existing audit log behaviour: when `audit.sink` is a GCPLOG connection, each expectation report should be sent directly to Google Cloud Logging instead. Use a separate, configurable log name for expectations (default `"expectations"`). The log entry payload must include all expectation fields (jobid, domain, schema, name, params, sql, count, exception, success) as JSON, with severity INFO and a label `type` set to `"expectation"`. Also ensure that existing audit log entries (not expectations) get a `type` label set to their step value. The existing SQL audit path for expectations should remain unchanged for other sink types.


**Title:** Wire BigQuery `clustering` property through ingestion, auto-job, audit, rejected, and metrics paths

**Description**
The `BigQueryLoadConfig` case class now accepts an `outputClustering: Seq[String]` parameter, but it is not yet populated from any user-facing configuration. Users should be able to control BigQuery table clustering by setting a `clustering` key in the schema/domain metadata properties (similar to the existing `timestamp` partition property). The value is expected to be a comma-separated list of column names.

**Requirements**
1. In the **ingestion job** (`IngestionJob`) and the **auto-job workflow** (`IngestionWorkflow`), when constructing a `BigQueryLoadConfig`, read the `"clustering"` key from the task’s `properties` map. Parse it into a `Seq[String]` splitting on `","`; if the key is absent, default to an empty sequence. Follow the same pattern already used for `"timestamp"` / `outputPartition`.
2. For internal tables that do not support user-defined clustering (**audit log**, **rejected records**, and **metrics**), explicitly pass an empty `Seq` (no clustering) when creating the `BigQueryLoadConfig`.
3. Ensure the change does not break any existing job that lacks a `clustering` property – absent key must behave identically to today.
4. No change to the YAML schema format or documentation is required in this ticket; the wiring alone is sufficient for the property to take effect end-to-end.

**Affected areas (for reference)**
- `IngestionJob.index` – normal data loading
- `IngestionUtil.indexRejected` – rejected records
- `SparkAuditLogWriter.append` – audit logs
- `MetricsJob.sinkMetricsToBigQuery` – metrics export
- `IngestionWorkflow.autoJob` – auto-job task indexing

**Expected outcome**
When a schema defines `clustering: "colA,colB"` in its metadata properties, the resulting BigQuery table will be created with those clustering columns. Tables created for audit, rejected, and metrics remain non-clustered.

Pass clustering param in BigQueryLoadConfig


**Title:** Roll back support for `kafka.properties` file

**Description**
We previously added experimental support for loading Kafka connection properties from a dedicated `kafka.properties` file placed in the metadata directory (loaded into `Settings.kafkaProperties`). This duplicated the existing `serverOptions` map already available in `KafkaConfig`. To avoid confusion and conflicting configuration sources, the feature is being removed.

**Expected behavior after the change:**
- The `kafka.properties` file is no longer read; no fallback loading logic from the metadata area should exist.
- The `Settings` instance must no longer carry a `kafkaProperties` field.
- All Kafka connectivity options (for the admin client, native consumers/producers, and Spark jobs) must now be obtained solely from `comet.kafka.serverOptions`.
- For Spark DataFrame reader/writer `.options()` calls, the Kafka-specific options need a `kafka.` prefix. However, the following keys must **not** be prefixed (they should be passed as-is):
  - `group.id`, `auto.offset.reset`, `key.deserializer`, `value.deserializer`, `enable.auto.commit`, `interceptor.classes`, `key.serializer`, `value.serializer`
  - Spark subscription keys: `assign`, `subscribe`, `subscribepattern`
- The `accessOptions` defined per topic should still be merged with the server-side properties via `allAccessOptions`, but the server-side source is now the `serverOptions` map (via whatever helper provides the Spark-compatible option set).
- Any existing code that previously used `Settings.kafkaProperties` directly (for admin client properties, consumer creation, or Spark options) must be updated to use the appropriate derived option map from `KafkaConfig`.

**Scope**
- Remove file-loading code and the `kafkaProperties` field.
- Centralize the prefixing logic for Spark Kafka options so that callers can obtain a `Map[String, String]` suitable for `.options()` from `serverOptions`.
- Update `KafkaClient` (admin client, consumer, producer, batch/stream read/write), `KafkaIngestionJob`, and the Kafka sink job to use the new source.
- Ensure the existing Kafka tests pass (they will be updated to reference the same logical configuration path).

**Acceptance criteria**
- `Settings` has no `kafkaProperties` member.
- No references to a `kafka.properties` file in production code.
- Streaming and batch Kafka jobs work correctly using only the `serverOptions` configuration, with the correct prefixing applied for Spark.

rollbakc support for file kafka.properties.


The `options` map on sink descriptors is never populated when reading YAML/JSON configuration – every sink ends up with an empty options map regardless of what is written in the config. Fix the Jackson deserialization so that the `options` property is correctly read from the input for all sink types (FS, BQ, ES, JDBC, None). Also ensure that an unknown sink type such as `KAFKA` is rejected rather than silently treated as some default. Once fixed, the `@JsonIgnore` workaround on `getOptions` should be removable.

add tests + try to fix options deserialization


Improve autocomplete so that it includes the internal system tables: audit log (`audit`), rejected records (`rejected`), and expectation reports (`expectations`). Currently `getObjectNames` only returns domains and tables from configuration files and externals. These three tables should appear under a domain whose name is read from `settings.appConfig.audit.domain`, defaulting to `"audit"`. Define the necessary schemas for these tables (matching the columns already used by the corresponding sink logic) and add the domain to the list returned by the autocomplete method.


**Title:** Transform command does not surface task failures as errors

**Problem**
When a `transform` job (including `--recursive`) encounters a failure during task execution (e.g., a SQL error in a transform step), the CLI process exits with a success status. There is no exception, and the calling environment (orchestrator, shell script) cannot detect that the job failed.

**Expected behavior**
- If a transform task fails for any reason, the `transform` command should cause the process to exit with a non‑zero exit code.
- The same behavior should hold for recursive transforms: if any task in the dependency chain fails, the whole job is considered failed.
- Exceptions thrown inside command execution should be caught and turned into a proper failure (not silently ignored).

**Additional context**
- The `IngestionWorkflow.autoJob` method that drives transformations returns `Try[Boolean]`, but it currently wraps a call to `transform(config)` without converting a `false` outcome into a failure. This means a task can return `false` (failure) and the `Try` still succeeds, hiding the error.
- The `Main` entry point that executes commands catches some parse errors but may not guard against exceptions thrown from the command implementations themselves.
- Currently integration tests treat `Main.run(...)` as fire‑and‑forget and do not assert its success; once fixed, callers will be able to rely on a Boolean success/failure indication or a thrown exception.


**Summary:** `.ack` files accidentally end up in the pending area; they should be ignored during ingestion and cleaned from the landing zone when domain configuration enables ack handling.

**Observed behavior:**
- When a data file (e.g., `data.csv`) is placed in the incoming landing area along with its readiness signal file (`data.ack`), the current ingestion workflow moves *both* files into the pending area.
- This pollutes the dataset with meaningless ack files and can cause downstream processing errors.

**Expected behavior:**
- `.ack` files must never be moved to the pending area – they are only a signal that a data file is fully written.
- When the domain descriptor (`domain.comet.yml`) includes an `ack` directive (indicating that ack-based cleanup is desired), the `.ack` file should be *deleted* from the landing area after the corresponding data file has been successfully moved to the pending area.
- Standard data files (`.csv`, `.json`, compressed variants) should continue to be imported correctly.
- The behavior should degrade gracefully when no ack configuration is present (ack files are simply not imported but may remain in the landing area unless explicitly cleaned up by the user).

**Impact / context:**
- This is part of the landing → staging → working zone pipeline.
- The fix must cover the normal file import path and any variant that operates on the landing zone (compressed files, raw files).
- Existing ingestion of data files without ack files must remain unaffected.

**Relevant configuration:**
- Domain YAML files already support an optional `ack` property; its presence should activate the deletion of ack files after a successful file move.


**Title:** BigQuery sink ignores custom `options` in configuration

**Background:**
`BigQueryLoadConfig` exposes an `options: Map[String, String]` field that should be forwarded to the Spark BigQuery connector when writing. However, these options are currently discarded—the write paths only set a handful of hard-coded options (`table`, `datePartition`, `intermediateFormat`). This prevents users from passing connector-specific settings (e.g., `allowFieldAddition`, `writeAtLeastOnce`, `partitionField`, etc.) that are needed for schema evolution or performance tuning.

Additionally, the audit-log writer hard-codes `Map("allowFieldAddition" -> "true")` instead of using the sink’s configured options, so any user-provided options are silently lost for audit tables as well.

**Desired behavior:**
- When `BigQuerySparkJob` writes data, it must apply all entries from `cliConfig.options` in addition to the mandatory options, both for the partitioned and non-partitioned code paths.
- When `SparkAuditLogWriter` creates a BigQuery job for audit logs, it must pass `sink.options` rather than a hard-coded map. The sink’s options should be combined with any necessary audit-specific defaults (e.g., `allowFieldAddition`), but the user-supplied options must take precedence if there are conflicts.

**Acceptance criteria:**
- Any option placed in the BigQuery sink configuration is effective when writing data tables.
- Audit-log BigQuery writes respect the sink’s options (merging with safe defaults as appropriate).
- No regression for existing configurations that do not set custom options (i.e., the change is purely additive).


**Bug Report: BigQuery Spark connector ignores configured location, defaults to US**

**Summary**
When a BigQuery connection specifies a non-US `location` (e.g., `EU`, `asia-northeast1`), the Spark BigQuery connector still performs internal operations — dataset lookups, query jobs — against the `US` multi-region. This causes failures when the target dataset resides in a different location.

**Observed behavior**
- The connection YAML includes `location: "EU"` (or any non-US value).
- `temporaryGcsBucket` and authentication from that same connection are correctly passed to the Spark session in `prepareConf`.
- The `location` option is present in `connectionOptions` but never reaches the Spark session configuration.
- Because the Spark BigQuery connector reads `location` from session-level config at runtime, it falls back to `US` for all internal BQ API calls, even though the user intended a different location.

**Expected behavior**
The `location` value from the BigQuery connection options should be propagated to the Spark session configuration, in the same way `temporaryGcsBucket` and auth are already propagated. The connector should then use the configured location for dataset lookups, query jobs, and any other location-sensitive operations.

**Impact**
Any user with BigQuery datasets outside the `US` multi-region cannot reliably use the Spark BigQuery connector through Starlake; their loads and transforms may hit spurious "Dataset not found" or location-mismatch errors from the BigQuery API.

**Notes**
The fix should handle the case where the connection options may or may not contain `location` (no-op when absent). The set of session properties already managed in the BigQuery Spark job's configuration logic is a clear precedent to follow.

Set location on Spark session in prepareConf for BigQuery connector


**Title:** Support Python commands in AutoJob tasks

**Description:**
Users currently define transformation tasks in AutoJob YAML files using SQL (`sql` field or standalone `.sql`/`.sql.j2` files). There is no equivalent for Python scripts. Teams want to run Python transformations as part of the same job DAG, with the same lifecycle (pre-script, post-script, sink, assertions, audit).

**Expected behavior:**

1. A task may specify a `python` attribute in the YAML, pointing to a Python file, or a `.py` file can be placed next to the job YAML using the same naming convention as SQL files (e.g., `<job>.<task>.py`).
2. At execution time, if the task defines a `python` file, the system should run it as a Spark Python script (using `PythonRunner`).
3. After the script finishes, if the Spark session contains a table named `STARLAKE_TABLE`, that table should be used as the task’s output DataFrame (and processed through sinks, assertions, post-scripts, and audit). If the table does not exist, the task should still succeed but with no output DataFrame.
4. Tasks must define exactly one of `sql` or `python`; defining both (or neither) is an error and should be caught during job validation.
5. The engine attribute that controls execution semantics (e.g., BigQuery vs. Spark) should be named `engine` (not `sqlEngine`), as it also applies to Python tasks. Backward-compatibility mapping for old `sqlEngine` YAML key can be provided.

**Motivation:**
Enables data engineers to embed Python transformations (UDFs, external libraries, ML models) directly into the Starlake job orchestration without leaving the declarative framework.

**Acceptance Criteria (conceptual):**
- An AutoJob task can be defined with a `python` file and runs correctly.
- The task validates that only one of `sql`/`python` is specified.
- Python task output is correctly handled (sinks, assertions, audit) depending on presence of `STARLAKE_TABLE`.
- The `.py` file discovery follows the same pattern as `.sql`/`.sql.j2` files.
- The `engine` attribute is renamed from `sqlEngine` (breaking change with compatibility mapping).


**Title:** BigQuery jobs do not respect project‑qualified dataset identifiers

**Description:**
Users can configure a BigQuery output dataset using a fully‑qualified identifier such as `my-project:my_dataset` or `my-project:my_dataset.my_table`. Currently, when a project ID is embedded in the dataset name, the BigQuery job paths ignore the project prefix and either fail or operate against the wrong project. This affects:

- ingestion merges from BigQuery (`mergeFromBQ`)
- native BigQuery load/create‑table flows
- Spark BigQuery load jobs
- application of access control policies and row‑level security

The fix must ensure that everywhere a BigQuery dataset or table identifier is constructed, the project ID is parsed from the input string and used when building the API object. If no project is present, the default project (from `fs.gs.project.id` or the service default) must be used as before.

**Acceptance criteria:**
- When `outputDataset` or a domain/schema name contains a project prefix (e.g. `project:dataset` or `project:dataset.table`), the job operates on that project.
- All BigQuery‑interacting code paths (Spark job, Native job, ingestion merge, access control) honour the project.
- Default‑project fallback continues to work when no project prefix is given.
- Resource identifiers using the colon format (project:`dataset.table`) and dot‑format (`dataset.table`) are correctly split into project, dataset, and table components.

fix: allow dataset rename on bigquery job


When defining an auto task with merge options, I need to customize the SQL based on whether the run is an initial load (target table does not exist) or a subsequent merge (target table already exists). I would like to use Jinja conditionals in the task SQL, such as `{% if merge %} ... {% endif %}`, but currently no `merge` variable is available.

Request:
- Expose a `merge` boolean in the Jinja context when a merge is actually happening. It should be `true` when the target table already exists **and** merge options are configured (i.e., the task will execute a MERGE statement), and `false` otherwise (including the initial load case).
- Make the Jinja template engine able to handle boolean values, not only strings, so that conditionals like `{% if merge %}` work correctly. This may require relaxing variable types from `String` to `Any` in the parsing utility, with automatic conversion to string only where substitution is needed.
- (Consistency) Rename the temporary view used in post-SQL processing and in Python task scripts from `SL_TABLE` to `SL_THIS`. This avoids confusion when merge workflows are in play and makes it clearer that the view represents the current task's output.

The `merge` variable should be available in the auto task's main SQL, as well as in any pre-SQL and post-SQL snippets associated with the task. It should be `false` during the initial load and after that `true` when the table exists and merge is configured.

Expose "merge" boolean when actual merge is happening in autotask


Implement these two changes to domain extension handling:

1. **Configurable default extensions**
   Introduce a configuration property `default-file-extensions` (a list of extensions without a leading dot, e.g. `["json", "csv", "dsv", "psv", "dat", "txt"]`). When a domain does not define its own `extensions`, use this list as the fallback. The old hard‑coded list `["json", "csv", "dsv", "psv"]` must not be hard‑coded in the domain model anymore.

2. **Normalize dot‑prefix**
   Regardless of whether the extensions come from the domain definition or the new default configuration, treat an extension with or without a leading dot as equivalent. For example, both `".csv"` and `"csv"` must match files whose names end with `".csv"`. The matching suffix should be `.<ext>` (a leading dot, then the extension without an extra dot). An extension that is already the empty string should remain as‑is (no dot added).

3. **Close a filtering gap in ingestion**
   When a domain does **not** require an ack file, the current logic skips extension filtering entirely: *every* file in the landing area is accepted. Correct this so that only files whose names end with one of the domain’s (or default) extensions are loaded, exactly as when an ack file is expected. This must work for both normal files and archive suffixes (e.g. `.zip`), keeping the existing `zipExtensions` check unchanged.

Add support fo default extensions to recognize and handle caases when extensions is defined with the dot prefix.


**Title:** Use BigQuery table IAM policies for access control instead of row-level security statements

**Context**
The BigQuery load job (`BigQuerySparkJob`) currently uses the `prepareRLS` method to generate and execute row‑level security SQL statements after loading a table. As a temporary workaround until BigQuery’s native row‑level security is available, we want to switch to setting IAM policies directly on the created table or view.

**Requested behavior**

1. **Model change**
   - The `RowLevelSecurity` case class should represent `grants` as a `Set[String]` (no duplicates).
   - The `predicate` field should default to `"TRUE"` so it becomes optional when only table‑level access is needed.

2. **Apply IAM policy after table creation**
   - After a BigQuery table is created or loaded, if the job configuration includes exactly one `rls` entry, set an IAM policy on the table that grants the **`bigquery.dataViewer`** role to every identity listed in `grants`.
   - The policy must only be set if the existing bindings differ from the desired ones, avoiding unnecessary API calls.

3. **Replace the old security path**
   - The existing `prepareRLS` / `runJob` logic inside the BigQuery load should be commented out (preserved for reference) and replaced by the new IAM policy application.

4. **Dependency bump**
   - The BigQuery client library may need a version upgrade to expose `getIamPolicy`/`setIamPolicy` on table resources. Update the dependency version accordingly.

**Notes**
- The `grants` strings follow the convention `"user:email"`, `"group:email"`, or `"serviceAccount:email"`.
- For now we expect exactly one `rls` definition per load; anything else should log a message and skip the policy application.
- This feature changes the security model from **row‑level filtering** to **table‑level viewer access** – document that BigQuery loads currently only support table‑level grants via IAM.

[FEATURE] - Table access controls on BigQuery (#356)


```
Feature: Support SQL files in views folder

Currently, views are defined in YAML configuration files (e.g., `*.comet.yml`) under the views area and referenced by name.
We want to support defining views directly as `.sql` files placed inside a dataset's or job's `views/` directory, without requiring YAML wrappers.

### Requirements

1. **Loading views from .sql files**
   - For a given view domain (e.g., an AutoTask or domain name), the system should scan the designated views directory recursively for files with `.sql` extension.
   - Each file's content is the SQL text of the view. The view name is derived from the file path relative to the views root, with path separators replaced by dots (`.`) and the `.sql` extension removed.
   - Example: file `views/CTE_daily_sales.sql` yields a view named `CTE_daily_sales`.
   - The legacy YAML‑based view loading should be deprecated; `.sql` files become the primary mechanism.

2. **Comment stripping**
   - SQL files (both view definitions and the main AutoTask SQL) may contain single‑line (`//`) and multi‑line (`/* … */`) comments.
   - These comments must be stripped before the SQL is assembled or executed.
   - Comment stripping should be applied to the view SQL when loaded and to the task's SQL before query construction.
   - If stripping fails (malformed comments), the system should raise a clear error.

3. **AutoTask query assembly – WITH clause handling**
   - Views are typically prepended as Common Table Expressions (CTE) using a `WITH ...` prefix to the task's main SQL.
   - **If the task's main SQL already starts with a `WITH` keyword** (case‑insensitive), the view CTEs should **not** be added; the task's SQL must be used as‑is (after comment stripping).
   - **Otherwise**, the existing behavior of wrapping view definitions as `WITH viewName AS ( ... )` and prefixing them to the main SQL must continue.

4. **BigQuery‑specific parenthesisation**
   - When the engine is BigQuery and the final, assembled query contains a `WITH` clause, the whole query must be wrapped in an extra pair of parentheses to satisfy BigQuery syntax requirements.
   - This applies regardless of whether the `WITH` came from the views or the task's own SQL.

### Expected outcome

A user can drop a `.sql` file into a `views/` folder, and it will automatically be available as a view inside the corresponding AutoTask without any YAML configuration. SQL comments are tolerated everywhere, and queries that already have their own `WITH` clause are handled gracefully without duplicate CTE sections.
```


**Title:** Jinja templating is skipped for YAML job files containing inline `sql:`

**Description**

When an auto job YAML file contains a `sql:` block (embedded SQL), Jinja template processing is bypassed. As a result any Jinja expressions in that file – e.g. `{{ my_var }}` – are left as literal text; the YAML is parsed with them unexpanded, causing the job to behave incorrectly or fail because the expressions are treated as ordinary strings.

The intended behaviour is that Jinja templating is applied uniformly to every job YAML file, **regardless of whether it contains an embedded `sql:` key**. Template variables should be replaced by their resolved values (from CLI `--options`, environment, etc.) *before* the YAML is parsed, exactly as it already works for job files without `sql:`.

**Acceptance criteria / constraints**

- Any YAML file loaded as an auto job definition must have its Jinja expressions processed (via `Utils.parseJinja`) before the YAML is interpreted, including files that contain a `sql:` key.
- The behaviour for job files without `sql:` must not change – they continue to receive Jinja expansion as before.
- The resolution context for Jinja variables in job files should be the same set of key‑value pairs that is used elsewhere when loading a job (derived from the active environment and any CLI‑provided options). There is no requirement to introduce a separate or special variable source solely for files with embedded SQL.

**Non‑goal**

- This change does **not** affect how Jinja or other templating is applied later during task execution (e.g., SQL templating inside `AutoTask`). The scope is restricted to the initial load and parsing of the YAML job description file.


**Title:** Add `--grantees` filter to `acl-dependencies` command for selective graph generation

**Description:**
The `acl-dependencies` command currently produces a full DOT graph of all ACL and Row-Level Security (RLS) relationships across domains, tables, and tasks. There is no way to restrict the output to only the grant relationships relevant to a specific set of users, groups, or service accounts.

We need a `--grantees` command-line option that accepts one or more grantee identifiers (e.g., `user:someone@example.com`, `group:engineering`, `serviceAccount:my-sa@project.iam.gserviceaccount.com`). When provided, the generated graph must include **only**:

- Users/groups/service accounts that appear in the `grantees` list.
- Tables, tasks, and domain groupings that directly grant ACL or RLS permissions to those grantees.
- Edges (relation arrows) that represent a grant to a listed grantee.

If the `--grantees` option is omitted or its list is empty, the command must behave exactly as before—generating the complete graph.

**Acceptance criteria / invariants:**
1. `acl-dependencies` (no arguments) → full graph, unchanged.
2. `acl-dependencies --grantees "user:alice@example.com,group:devs"` → only entities and relations involving `user:alice@example.com` or `group:devs`.
3. Filtering applies both to table-level ACL/RLS grants and to task-level ACL/RLS grants.
4. The existing CLI interface (including `--outputFile` and `--reload`) is unchanged except for the new option.
5. No regressions in the unfiltered output.

**Notes:**
- A grantee may appear in an `acl` list or an `rls` definition on a table or task.
- The graph generation must keep the same structural format; only the set of included nodes and edges should shrink.
- The `grantees` list represents case‑sensitive identifiers as they appear in the YAML definitions.

Let me know if there are any questions about the expected semantics.

Generate dependencies graphs on ACL selectively


# Bug Report: HTML report duplicates test's own trace as a Linked Trace

The HTML reporter incorrectly shows the test's own trace identifier twice: once in the **Trace Timeline** section (correct) and once under **Linked Trace: TUnit** (incorrect). Only traces explicitly registered by users via `TestContext.RegisterTrace` should appear as Linked Traces; the test's primary trace should not be listed there.

## Steps to reproduce
1. Run any test that has a trace context enabled (for example, through OpenTelemetry instrumentation).
2. Optionally call `TestContext.RegisterTrace(anotherTraceId)` to add one or more external trace IDs.
3. Open the generated HTML report and locate the test's result entry.
4. Observe the **Linked Traces** list.

## Current behavior
The test's own `traceId` appears in the Linked Traces list alongside any user-registered traces. This causes the primary trace to be displayed twice: filtered through the Trace Timeline and again as a raw Linked Trace entry.

## Expected behavior
- The **Linked Traces** list should contain only trace IDs that were explicitly added by the user via `TestContext.RegisterTrace`.
- The primary trace ID (the one already rendered in the Trace Timeline) must never appear in the Linked Traces list.
- Matching of trace IDs should be case-insensitive, as trace IDs from different sources may vary in casing.
- If the primary trace ID is not available (null), all registered trace IDs should appear as linked traces (unchanged legacy behavior).
- If the only registered trace ID is the primary one (no user-added traces), the Linked Traces list should be empty and not rendered.

## Impact
The duplication is confusing and breaks the intended separation between the test's own execution trace and externally correlated traces. It also suggests to users that they accidentally registered the test's own trace, while it actually comes from the engine's automatic OTLP correlation registration.

## Additional context
The engine automatically registers the test's own `traceId` into the `TraceRegistry` for OTLP cross-process log correlation. The HTML reporter currently reads all entries from `TraceRegistry` without filtering out the primary trace, causing the duplication. The fix should ensure the reporter excludes the primary trace when constructing the additional-trace list.

fix: don't render test's own trace as Linked Trace in HTML report (#5580)


## AspireFixture times out on Aspire 13.2.0+ because it waits on internal child resources that never become healthy

**Component:** `TUnit.Aspire` integration — the `AspireFixture<T>` lifecycle that waits for resources to become healthy/running before test execution.

**Symptom**
When using the `AspireFixture` with Aspire SDK >= 13.2.0, tests that pass on earlier Aspire versions start hanging and eventually throw a `TimeoutException`. The timeout occurs because the fixture is waiting on resources that never emit a healthy or running state.

**Root cause**
Aspire 13.2.0 introduced a new internal resource type (`ProjectRebuilderResource`) that implements `IComputeResource` **and** `IResourceWithParent`. This resource is a child resource managed by the parent project and never becomes healthy on its own. The fixture’s current wait logic selects all `IComputeResource` resources unconditionally, so this resource gets included and blocks the wait indefinitely.

**Expected behavior**
The waitable-resource selection should exclude resources that are not meant to be independently awaited:

1. **`IComputeResource` that also implements `IResourceWithParent`**
   These are Aspire-managed children (e.g., the rebuild resource). They cannot be awaited directly and must be excluded.

2. **Non-compute resources** (`IResource` that are not `IComputeResource`, like `ParameterResource`, `ConnectionStringResource`)
   These were already excluded (they never report healthy), so that behavior must be preserved.

3. **Resources marked `IsHidden` in DCP snapshots** (runtime safety net)
   Even if a resource slips through the model-level filter, if its runtime snapshot has `IsHidden == true` it should be skipped at the point of waiting. The fixture must not wait on hidden resources.

**Constraints & invariants**

- **Do not change behavior for normal `IComputeResource` resources** (containers, projects, executables). They must still be waited on and must not regress.
- The fix must work correctly when multiple resources of different types are present; filtering must not accidentally drop legitimate compute resources.
- The rejection of hidden resources should be applied at runtime (during the wait loop) to catch edge cases where Aspire registers hidden resources that the model-level filter misses.
- The filter logic should be **customizable** by users: provide an extension point (e.g., a `protected virtual` method) so that tests can opt additional resources in or out of the wait set without modifying the framework.

**Acceptance criteria / how to verify**
- With Aspire 13.2.0+, a fixture that starts an application with a project resource no longer times out waiting for `ProjectRebuilderResource`.
- Explicitly verifying the wait-set selection logic shows that:
  - A plain `IComputeResource` (container/project) is included.
  - A resource implementing both `IComputeResource` and `IResourceWithParent` is excluded.
  - A non-`IComputeResource` (e.g., parameter) is excluded.
- A custom subclass of `AspireFixture<T>` can override the selection predicate to further exclude custom resources without breaking the default behavior.

---

**Suggested approach (not prescriptive)**
The selection logic currently lives in the method that builds the list of resource names to wait on. Consider extracting the decision “should this resource be waited for?” into a dedicated virtual method so it can be overridden, and then use that both when building the initial list and when performing runtime hidden-resource filtering.

Fix Aspire 13.2.0+ timeout caused by ProjectRebuilderResource being awaited (#5335)


**Title**: JUnit XML reporter crashes when test names or exceptions contain invalid XML characters

**Context**
When the JUnit XML reporter writes a test results file, it uses `System.Xml.XmlWriter` to build the document. If any string value passed to the writer (test name, class name, error message, exception `ToString()`, skipped reason, etc.) contains a character that is not legal in XML 1.0, `XmlWriter` throws an exception. This prevents the entire report from being generated, even if only one test case is affected.

The problem surfaces most often when:
- Test names or parameterised argument values include binary/control characters (e.g., `\x04`, `\x1F`).
- Exception messages or stack traces contain such characters, for example when a test fails on data that includes them.

**Expected behaviour**
The JUnit XML output must always be well-formed and the reporter must never crash due to content. Any string written into the XML should be sanitised so that it contains only valid XML 1.0 characters. The sanitisation must preserve the standard XML-legal control characters tab (`\t`), line feed (`\n`), and carriage return (`\r`), since those are explicitly allowed. All other invalid codepoints should be replaced with a deterministic, visible placeholder (for example `[0xNN]` where NN is the hexadecimal code point) so that they can be identified and not silently lost.

This sanitisation must be applied thoroughly to every free-text value that ends up in an XML attribute or text node, including (but not limited to):
- Assembly name
- Hostname
- Test name (the `name` attribute on `<testcase>`)
- Class name (`classname` attribute)
- Exception messages, types, and full `ToString()` output in `<failure>` and `<error>` elements
- Skipped and timeout messages
- Property values (framework, platform, runtime, filter)

A null or empty input should be handled gracefully and result in an empty string.

**Acceptance Criteria**
- Running a test suite that exercises the scenario above (test names/exception messages with invalid XML characters) produces a valid JUnit XML report without throwing.
- The invalid characters are replaced with the placeholder representation, preserving as much readable text as possible.
- Valid XML control characters (`\t`, `\n`, `\r`) are passed through unchanged.
- Normal test runs with no unusual characters are unaffected (the report output is identical to before).

Fix JUnit reporter crash on invalid XML characters in test names and exceptions (#4262)


**Title:** Fix AwaitAssertionAnalyzer false positives and spurious warnings

**Background**
The `AwaitAssertionAnalyzer` in `TUnit.Assertions.Analyzers` is responsible for flagging calls to `Assert.That` and `Assert.Multiple` that are not awaited. It currently generates incorrect warnings (false positives) in several scenarios.

**Observed problems**

1. **Await detection too shallow**
   The analyzer only looks at the immediate children of the containing expression statement when searching for an `await` expression. If the `await` is nested inside a multi-step fluent chain, a lambda, or any other non-trivial child structure, the analyzer misses it and still reports a missing await.
   *Expected behavior*: If there is *any* `await` expression anywhere within the expression statement, the call is already awaited and should not be flagged.

2. **Method name comparison based on syntax text**
   The analyzer compares method names using `memberAccessExpressionSyntax.ToString()` against string literals like `"Assert.That"` and `StartsWith("Assert.Multiple")`. This breaks with:
   - Fully qualified invocations (`global::TUnit.Assertions.Assert.That(...)`)
   - Aliased imports or `using static` that change the syntactic appearance
   - Helper wrapper methods that delegate to `Assert.That` (though the primary concern is direct calls appearing under different syntax)
   *Expected behavior*: Use the semantic model to resolve the method symbol, then compare its fully qualified, non-generic name against the canonical TUnit assertion methods `global::TUnit.Assertions.Assert.That` and `global::TUnit.Assertions.Assert.Multiple`.

3. **Inside `Assert.Multiple` exemptions**
   The analyzer currently exempts `Assert.That` calls that appear inside an `Assert.Multiple` block from the missing-await diagnostic. This exemption is also done via string comparison on the syntax node, leading to incorrect exemptions (or lack thereof) when the `Assert.Multiple` call is written with a non-standard syntax.

**Acceptance criteria**

- No diagnostic is reported when the expression statement contains an `await` expression at *any* depth.
- Method identity checks are performed symbolically, using the semantic model’s fully qualified method name, so that aliases, full qualification, and `using static` do not affect correctness.
- The exemption for `Assert.That` invocations that are already inside an `Assert.Multiple` block continues to work correctly under all naming styles.
- All existing false positives reported by the team are resolved; specifically, patterns with fully qualified method calls and nested awaits no longer produce spurious warnings.

**Non-requirements**
- No changes to the diagnostic messages or diagnostic IDs.
- No changes to the overall architecture of the analyzer; only the analysis logic in `AwaitAssertionAnalyzer` should be adjusted.


### Use stack-allocated buffer and remove unnecessary async state machines in `EventReceiverOrchestrator` hot paths

The `EventReceiverOrchestrator` methods are called for every test during discovery and execution, and they currently introduce avoidable allocations and async overhead that affect large test suites.

**What needs to change:**

1. **`RegisterReceivers`**
   It collects eligible event objects from a `TestContext` and passes them to `EventReceiverRegistry.RegisterReceivers`. Currently it defers a `List<object>` allocation until needed, then adds to it. Replace this with a stack-allocated builder (such as `ValueListBuilder<object>`) so that in the common case of few objects we avoid any heap allocation. After building, pass the data as a `ReadOnlySpan<object>` to the registry method, which should be updated to accept a span instead of `IEnumerable<object>`.

2. **`InvokeTestDiscoveryEventReceiversAsync`**
   This method currently always awaits inside an `async ValueTask` method. If there are no discovery receivers, return a completed `Task` synchronously without an async state machine. When there *are* receivers, extract the actual await loop into a separate private static `async Task` method so that the fast‑path does not incur the state-machine allocation, and the signature can become `Task` (not `ValueTask`).

3. **`InvokeTestEndEventReceiversAsync`**
   Add an early return inside this method: if no test‑end receivers exist for the requested stage (or for *both* `Early` and `Late` when no stage is specified), return an empty `ValueTask<IReadOnlyList<Exception>>` immediately without invoking the inner implementation.

4. **`TestBuilder.InvokeDiscoveryEventReceiversAsync`**
   Remove the `async`/`await` and simply return the `Task` from the orchestrator call, avoiding an extra state machine.

**Safety/behavior constraints:**
- The event dispatch semantics must remain identical – all receivers that would previously have been called must still be called, in the same order.
- The empty‑receiver fast‑paths must not accidentally skip registration or invocation when receivers *are* present.
- The stack‑allocated buffer must be properly disposed; if the builder pattern is used, ensure it is not leaked.
- The changes must work across the target frameworks (e.g., net6.0+), respecting any conditional compilation that exists.

The goal is to reduce per‑test allocations and method‑call overhead without altering any observable test behavior.

perf: use `VLB` and make methods sync in `EventReceiverOrchestrator` (#4410)