multihop_tasks_walkthrough.md

# Multihop Temporal Reasoning Tasks — Detailed Walkthrough

## Architecture Overview

All 8 new task generators follow a shared architecture built on `MultihopBaseGenerator`, which provides:
- **Scene building**: Constructing multi-event audio sequences with controlled order, durations, volumes, and silences
- **Dataset generation**: Sample-loop orchestration with balanced question type distribution
- **CSV output**: Standardized MCQ, open-text, and metadata CSV writing

```mermaid
graph TD
    A[MultihopBaseGenerator] --> B[ConditionalCountTaskGenerator]
    A --> C[ConditionalDurationTaskGenerator]
    A --> D[BetweenEventsTaskGenerator]
    A --> E[EventDensityTaskGenerator]
    A --> F[DurationGapTaskGenerator]
    A --> G[TemporalArithmeticTaskGenerator]
    A --> H[TemporalLoudnessTaskGenerator]
    A --> I[MultiHopTaskGenerator]
    
    J[ESC50Dataset] --> A
    K[PreprocessedESC50Dataset] --> C
    K --> F
    K --> G
    K --> I
    L[AudioProcessor] --> A
    M[QuestionGenerator] --> A
```

### Key Design Decision: Why a Shared Base?

The existing 7 tasks (count, duration, order, etc.) are standalone classes with no shared base. This worked because each task has fundamentally different audio construction logic (e.g., overlap uses `overlay()`, during_contains uses temporal containment, silence_gap uses controlled gaps).

The **multihop tasks are different** — they all build the same kind of audio scene (sequential events with gaps and volume variation) but ask different questions over that scene. The complexity is in the **question generation and answer computation**, not the audio construction. So a shared base class makes sense here.

---

## Base Class: `MultihopBaseGenerator`

**File:** [multihop_base.py](file:///home/debarpanb1/TREA_2.0/pipeline/tasks/multihop_base.py)

### What it provides

| Method | Purpose |
|--------|---------|
| `__init__` | Loads config, initializes dataset/processor/question generator, creates output dirs |
| `build_sequential_scene()` | Builds an audio scene with N events, controlled gaps, optional per-event volume |
| `generate_dataset()` | Orchestrates sample generation loop with balanced question types |
| `_save_mcq_csv()` | Writes MCQ CSV in standard pipeline format |
| `_save_open_text_csv()` | Writes open-text CSV in standard pipeline format |
| `_save_metadata_csv()` | Writes metadata CSV |

### Scene Building Logic

```
┌─────────┐  gap_0  ┌─────────┐  gap_1  ┌─────────┐  gap_2  ┌─────────┐
│ Event 0 │◄──────►│ Event 1 │◄──────►│ Event 2 │◄──────►│ Event 3 │
│  cat_A   │         │  cat_B   │         │  cat_A   │         │  cat_C   │
│  vol: 0dB│         │ vol:-6dB │         │ vol:+3dB │         │ vol:-2dB │
└─────────┘         └─────────┘         └─────────┘         └─────────┘
```

1. Sample `n_events` unique categories from ESC-50
2. Load and extend each source clip to fill its time slot via `concatenate_to_target_duration()`
3. Apply per-event volume adjustments (if specified)
4. Generate controlled gap durations using `generate_controlled_gap_durations()` (ensures the longest gap is ≥ `gap_multiplier × shortest gap`)
5. Concatenate: `event + silence + event + silence + ...`
6. Return per-event metadata (start_ms, end_ms, duration, volume, category)

---

## Task 1: `ConditionalCountTaskGenerator`

**File:** [task_conditional_count.py](file:///home/debarpanb1/TREA_2.0/pipeline/tasks/task_conditional_count.py)
**Family:** Multihop Singular
**Reasoning chain:** Temporal condition → Count

### What it does

Counts sound events **after applying a temporal filter**. Unlike the simple COUNT task which asks "how many unique sounds?", this task asks "how many X sounds occur **after** Y?" — requiring the model to first identify when Y occurs, then count X events only in the filtered region.

### Audio Construction

Builds a scene with **4–8 events**, intentionally allowing **repeated categories** (unlike between_events which uses all unique). This is critical because counting repeated occurrences is the core question.

```python
# Example scene: categories = [dog, cat, dog, bird, cat, dog, bird]
# count_after(target=dog, anchor=cat[0]) → 3 (indices 2, 5 after first cat)
# count_before(target=bird, anchor=dog[2]) → 1 (index 3 before third dog)
```

### Question Types & Answer Computation

| Type | Logic |
|------|-------|
| `count_after` | Pick anchor index → count `target_sound` in `categories[anchor+1:]` |
| `count_before` | Pick anchor index → count `target_sound` in `categories[:anchor]` |
| `count_between` | Pick two anchor indices → count events in `categories[idx1+1:idx2]` |
| `count_during` | Pick anchor → count all other `target_sound` occurrences |
| `count_overlap` | Pick anchor → count adjacent events (neighbors at ±1 index) |

### MCQ Generation

Uses `QuestionGenerator.generate_count_mcq()` — generates 4 integer options including the correct count, with distractors sampled from nearby integers.

---

## Task 2: `ConditionalDurationTaskGenerator`

**File:** [task_conditional_duration.py](file:///home/debarpanb1/TREA_2.0/pipeline/tasks/task_conditional_duration.py)
**Family:** Multihop Singular
**Reasoning chain:** Temporal condition → Duration comparison

### What it does

Compares durations **only among events in a temporal subset**. "Which sound **after** Y lasts the longest?" requires: (1) identify Y's position, (2) consider only sounds after Y, (3) compare their durations.

### Audio Construction

Uses **`PreprocessedESC50Dataset`** (not plain ESC50Dataset) to get **effective durations** — the actual content duration after silence trimming. This ensures duration comparisons are meaningful (a 5s clip with 3s of trailing silence has effective duration 2s).

```python
# Each event loaded with known effective duration:
fname, fpath, eff_dur = self.dataset.sample_file_from_category_with_duration(cat)
```

### Question Types & Answer Computation

| Type | Logic |
|------|-------|
| `longest_after` | Filter events after anchor → `max(effective_durations)` |
| `shortest_after` | Filter events after anchor → `min(effective_durations)` |
| `longest_before` | Filter events before anchor → `max(effective_durations)` |
| `shortest_before` | Filter events before anchor → `min(effective_durations)` |
| `repeated_compare` | Find category appearing both before AND after anchor → compare their durations |

### MCQ Generation

Uses `QuestionGenerator.generate_category_mcq()` — the answer is a sound category name, with distractors drawn from other categories present in the audio.

---

## Task 3: `BetweenEventsTaskGenerator`

**File:** [task_between_events.py](file:///home/debarpanb1/TREA_2.0/pipeline/tasks/task_between_events.py)
**Family:** Multihop Singular
**Reasoning chain:** Define temporal window → Identify/Count events inside

### What it does

Identifies or counts sounds occurring **inside the temporal window between two anchor events**. "Which sound occurs between X and Y?" requires: (1) find X, (2) find Y, (3) identify events strictly between them.

### Audio Construction

Uses **all unique categories** (no repeats) so that "which sound between X and Y?" has an unambiguous answer. Scene has **5–8 events**.

### Question Types & Answer Computation

| Type | Logic |
|------|-------|
| `identify_between` | Pick two anchors `idx1 < idx2` → randomly select one event from `categories[idx1+1:idx2]` as answer |
| `count_between` | Pick two anchors → `len(categories[idx1+1:idx2])` |
| `yes_no_between` | Pick target → check if target is in the between-window (50/50 true/false balance) |

### Key Constraint

Anchors are chosen with **at least 2 positions apart** (`idx2 ≥ idx1 + 2`) to guarantee at least 1 event between them. Samples with fewer than 3 events are rejected.

---

## Task 4: `EventDensityTaskGenerator`

**File:** [task_event_density.py](file:///home/debarpanb1/TREA_2.0/pipeline/tasks/task_event_density.py)
**Family:** Multihop Singular
**Reasoning chain:** Segment audio into regions → Compare event counts

### What it does

Compares **event frequency across temporal regions**. "Which half of the audio contains more sound events?" requires counting events in each half and comparing.

### Audio Construction

Builds scenes with **6–10 events** (larger scenes to make density differences meaningful). **Allows repeated categories** so label-specific density questions are possible. Categories are shuffled randomly — the natural random distribution creates asymmetry.

### Question Types & Answer Computation

| Type | Logic |
|------|-------|
| `half_density` | Split at midpoint → compare `len(first_half)` vs `len(second_half)` → answer is "first half" / "second half" / "equal" |
| `before_after_density` | Pick anchor → compare count of events before vs after anchor |
| `label_density` | Pick anchor + target label → compare count of `target_sound` before vs after anchor |

### MCQ Generation

Custom option construction — options are textual ("first half", "second half", "equal", "cannot determine") rather than category names.

---

## Task 5: `DurationGapTaskGenerator`

**File:** [task_duration_gap.py](file:///home/debarpanb1/TREA_2.0/pipeline/tasks/task_duration_gap.py)
**Family:** Multihop Inter-Task
**Reasoning chain:** Duration reasoning ↔ Silence-gap reasoning

### What it does

**Crosses two task families** — compares sound event duration against silence gap duration. "Which is longer: the dog sound or the silence after the dog sound?" requires perceiving both the event's length and the gap's length, then comparing.

### Audio Construction

Uses `PreprocessedESC50Dataset` for effective durations. **Wider gap range** (500–4000ms) compared to other tasks, ensuring gaps are sometimes longer and sometimes shorter than events — making the comparison non-trivial.

Each event's metadata tracks both `effective_duration_ms` and `gap_before_ms` / `gap_after_ms`.

### Question Types & Answer Computation

| Type | Logic |
|------|-------|
| `event_vs_after_gap` | Pick event (not last) → compare `effective_duration_ms` vs `gap_after_ms` |
| `event_vs_before_gap` | Pick event (not first) → compare `effective_duration_ms` vs `gap_before_ms` |
| `gap_vs_event` | Pick a gap between events A,B → pick third event C → compare `gap_duration` vs `C.effective_duration_ms` |

---

## Task 6: `TemporalArithmeticTaskGenerator`

**File:** [task_temporal_arithmetic.py](file:///home/debarpanb1/TREA_2.0/pipeline/tasks/task_temporal_arithmetic.py)
**Family:** Multihop Inter-Task
**Reasoning chain:** Duration + Count/Repetition + Silence reasoning

### What it does

Compares **aggregated** durations — total active time of one label vs another, or total sound time vs total silence time. Requires mentally summing up all occurrences of a repeated sound.

### Audio Construction

Uses `PreprocessedESC50Dataset`. **Allows repeated categories** (critical for aggregation). Computes two aggregate quantities:
- **`label_totals`**: Total effective duration per category (summed across all occurrences)
- **`total_silence_ms`**: Sum of all gap durations

### Question Types & Answer Computation

| Type | Logic |
|------|-------|
| `total_label_duration` | Pick two categories → compare `label_totals[sound1]` vs `label_totals[sound2]` |
| `label_vs_silence` | Pick one category → compare `label_totals[target]` vs `total_silence_ms` |
| `combined_duration` | Pick one category → compare `label_totals[target]` vs `max(gap_durations)` (longest single silence) |

---

## Task 7: `TemporalLoudnessTaskGenerator`

**File:** [task_temporal_loudness.py](file:///home/debarpanb1/TREA_2.0/pipeline/tasks/task_temporal_loudness.py)
**Family:** Multihop Inter-Task
**Reasoning chain:** Temporal filter → Loudness comparison

### What it does

Combines **temporal filtering** with **loudness comparison**. "Which sound after Y is the loudest?" requires: (1) identify Y's position, (2) consider only sounds after Y, (3) compare their loudness levels.

### Audio Construction

Each event gets a **random volume adjustment** from `volume_range_db` (default: [-12, +6] dB). A minimum volume difference (`min_volume_diff_db`: 3dB) is enforced to ensure the loudest/softest distinction is perceivable.

Actual loudness is measured using **LUFS** (`get_lufs_loudness()`) for perceptually accurate loudness comparison.

### Question Types & Answer Computation

| Type | Logic |
|------|-------|
| `loudest_after` | Filter events after anchor → `max(loudness_lufs)` |
| `softest_after` | Filter events after anchor → `min(loudness_lufs)` |
| `loudest_before` | Filter events before anchor → `max(loudness_lufs)` |
| `softest_before` | Filter events before anchor → `min(loudness_lufs)` |
| `repeated_loudness` | Find category with multiple occurrences → which occurrence is loudest? Also detects trend (louder/softer over time) |

---

## Task 8: `MultiHopTaskGenerator`

**File:** [task_multi_hop.py](file:///home/debarpanb1/TREA_2.0/pipeline/tasks/task_multi_hop.py)
**Family:** Multihop Inter-Task
**Reasoning chain:** Explicit 2–3 step chains across different properties

### What it does

The **most complex task** — requires explicit multi-step reasoning across different temporal/acoustic properties. "What sound occurs after the longest sound?" requires: (1) identify the longest sound (duration property), (2) find what comes after it (order property).

### Audio Construction

The **richest scene** — combines ALL controllable dimensions:
- **Duration variation** via PreprocessedESC50Dataset effective durations
- **Volume variation** via per-event dB adjustments ([-10, +6] dB range)
- **Silence variation** via controlled gaps (500–3000ms range, 2.5× multiplier)
- **Category repetition** for count-based sub-questions

### Question Types & Reasoning Chains

| Type | Step 1 | Step 2 | Step 3 |
|------|--------|--------|--------|
| `after_longest` | Find longest duration event | Return next event in sequence | — |
| `before_longest` | Find longest duration event | Return previous event | — |
| `after_shortest` | Find shortest duration event | Return next event | — |
| `before_loudest` | Find loudest event (LUFS) | Return previous event | — |
| `after_longest_gap` | Find longest silence gap | Return event after that gap | (or count remaining events) |
| `loudest_after_anchor` | Find anchor position | Filter events after anchor | Find loudest among them |
| `longest_before_anchor` | Find anchor position | Filter events before anchor | Find longest among them |
| `count_before_loudest` | Find loudest event | Filter events before it | Count target_sound occurrences |
| `count_after_longest` | Find longest event | Filter events after it | Count target_sound occurrences |

### Answer Computation Example

```python
# after_longest: "What sound occurs after the longest sound?"
pivot_idx = durations_ms.index(max(durations_ms))  # Step 1: find longest
target_idx = pivot_idx + 1                          # Step 2: get next event
correct = categories[target_idx]                    # Answer: that category
```

---

## Summary: What Makes Each Task Unique

| Task | Audio Uniqueness | Question Uniqueness |
|------|-----------------|-------------------|
| **conditional_count** | Repeated categories | Temporal filter → integer count |
| **conditional_duration** | PreprocessedESC50 effective durations | Temporal filter → duration superlative |
| **between_events** | All unique categories, ≥3 events | Temporal window identification |
| **event_density** | 6–10 events, asymmetric distribution | Region-based count comparison |
| **duration_gap** | Wide gap range (500–4000ms) | Cross-property: event-dur vs silence-dur |
| **temporal_arithmetic** | Repeated categories + tracked totals | Aggregated duration arithmetic |
| **temporal_loudness** | Per-event volume variation + LUFS measurement | Temporal filter → loudness superlative |
| **multi_hop** | All dimensions varied simultaneously | 2–3 step reasoning chains |